Eclector User’s Manual ¶

2.1.1 Package for basic features ¶

The package for basic features such as customizable source location construction is named eclector.base. Although this package does not shadow any symbol in the common-lisp package, we still recommend the use of explicit package prefixes to refer to symbols in this package.

2.1.2 Package for ordinary reader features ¶

The package for ordinary reader features is named eclector.reader. To use features of this package, we recommend the use of explicit package prefixes, simply because this package shadows and exports names that are also exported from the common-lisp package. Importing this package will likely cause conflicts with the common-lisp package otherwise.

2.1.3 Package for readtable features ¶

The package for readtable-related features is named eclector.readtable. To use features of this package, we recommend the use of explicit package prefixes, simply because this package shadows and exports names that are also exported from the common-lisp package. Importing this package will likely cause conflicts with the common-lisp package otherwise.

2.1.4 Package for parse result construction features ¶

The package for features related to the creation of client-defined parse results is named eclector.parse-result. To use features of this package, we recommend the use of explicit package prefixes, simply because this package shadows and exports names that are also exported from the common-lisp package. Importing this package will likely cause conflicts with the common-lisp package otherwise.

2.1.5 Package for CST features ¶

The package for features related to the creation of concrete syntax trees is named eclector.concrete-syntax-tree. To use features of this package, we recommend the use of explicit package prefixes, simply because this package shadows and exports names that are also exported from the common-lisp package. Importing this package will likely cause conflicts with the common-lisp package otherwise.

2.3.1 Common Lisp reader compatible interface ¶

The following functions are like their standard Common Lisp counterparts with the two differences that their names are symbols in the eclector.reader package and that their behavior can deviate from that of the standard reader depending on the value of the eclector.base:*client* variable.

Function: read [eclector.reader] &optional (input-stream *standard-input*) (eof-error-p t) (eof-value nil) (recursive-p nil) ¶

This function is the main entry point for the ordinary reader. It is entirely compatible with the standard Common Lisp function cl:read.

Function: read-preserving-whitespace [eclector.reader] &optional (input-stream *standard-input*) (eof-error-p t) (eof-value nil) (recursive-p nil) ¶

This function is entirely compatible with the standard Common Lisp function read-preserving-whitespace.

Function: read-from-string [eclector.reader] string &optional (eof-error-p t) (eof-value nil) &key (start 0) (end nil) (preserve-whitespace nil) ¶

This function is entirely compatible with the standard Common Lisp function cl:read-from-string.

Function: read-delimited-list [eclector.reader] char &optional (input-stream *standard-input*) (recursive-p nil) ¶

This function is entirely compatible with the standard Common Lisp function cl:read-delimited-list.

2.3.2 Reader behavior protocol ¶

By defining methods on the generic functions of this protocol, clients can customize the high-level behavior of the reader.

Figure 2.2: Functions and typical function call sequences terminating on the right hand side of the diagram at customizable generic functions which implement aspects of the reader algorithm and standard reader macros. Solid arrows represent calls, dashed arrows represent returns from function calls. Labels above arrows represent arguments and return values.

Figure 2.2 illustrates how the customizable generic functions described in this section are called through the client interface and the implementation of the reader algorithm.

Generic Function: call-as-top-level-read [eclector.reader] client thunk input-stream eof-error-p eof-value preserve-whitespace-p ¶

This generic function is called by eclector.reader:read if read is called with a false value for the recursive-p parameter. It calls thunk with the necessary context for a global read call. thunk should read and return an object without consuming any whitespace following the object. If preserve-whitespace-p is false, this function reads up to one character of whitespace after thunk returns. By default, this function returns the object or eof-value returned by thunk as its sole value.

Note: This generic function may return more values in addition to the one described above. Clients may use this feature to communicate additional information between methods (see Parse result construction features). Client defined methods on this generic function should accept such additional values when calling thunk, a next method or eclector.reader:read-common and themselves return the additional values.

The default method on this generic function performs two tasks:

1. It establishes a context in which labels (#N=) and references (#N#) work.
2. It realizes the requested preserve-whitespace-p behavior.

Generic Function: read-common [eclector.reader] client input-stream eof-error-p eof-value ¶

This generic function is called by eclector.reader:read, passing it the value of the variable eclector.base:*client* and the corresponding parameters. By default, this generic function returns the objects as its sole value.

Note: This generic function may return more values in addition to the one described above. Clients may use this feature to communicate additional information between methods (see Parse result construction features). Client defined methods on this generic function should accept such additional values when calling a next method or read-maybe-nothing and themselves return the additional values.

Client code can add methods on this function, specializing them to the client class of its choice. The actions that eclector.reader:read needs to take for different values of the parameter recursive-p have already been taken before read calls this generic function.

Generic Function: read-maybe-nothing [eclector.reader] client input-stream eof-error-p eof-value ¶

This generic function can be called directly by the client or by the generic function eclector.reader:read-common to read an object or consume input without returning an object. If called directly by the client, the call has to be in the dynamic scope of a eclector.reader:call-as-top-level-read call. The function read-maybe-nothing either

• encounters the end of input on input-stream and, depending on eof-error-p either signals an error or returns the values eof-value and :eof
• or reads one or more whitespace characters an returns the values nil and :whitespace
• or reads an object. If *read-suppress* is true, the function returns nil and :suppress. Otherwise it returns the object and :object.
• or consumes a macro character and the characters consumed by the associated reader macro function if that reader macro function does not return a value. In this case the function returns nil and :skip.

Note: This generic function may return more values in addition to the ones described above. Clients may use this feature to communicate additional information between methods (see Parse result construction features). Client defined methods on this generic function should accept such additional values when calling a next method and themselves return the additional values.

Generic Function: note-skipped-input [eclector.reader] client input-stream reason ¶

This generic function is called whenever the reader skips some input such as a comment or a form that must be skipped because of a reader conditional. It is called with the value of the variable eclector.base:*client*, the input stream from which the input is being read and an object indicating the reason for skipping the input. The default method on this generic function does nothing. Client code can supply a method that specializes to the client class of its choice.

When this function is called, the stream is positioned immediately after the skipped input. Client code that wants to know both the beginning and the end of the skipped input must remember the stream position before the call to eclector.reader:read was made as well as the stream position when the call to this function is made.

Variable: *skip-reason* [eclector.reader] ¶

This variable is used by the reader to determine why a range of input characters has been skipped. To this end, internal functions of the reader as well as reader macros can set this variable to a suitable value before skipping over some input. Then, after the input has been skipped, the generic function eclector.reader:note-skipped-input is called with the value of the variable as its reason argument.

As an example, the method on eclector.reader:note-skipped-input specialized to eclector.parse-result:parse-result-client relays the reason and position information to the client by calling the eclector.parse-result:make-skipped-input-result generic function (see Parse result construction features).

Generic Function: read-token [eclector.reader] client input-stream eof-error-p eof-value ¶

This generic function is called by eclector.reader:read-common when it has been detected that a token should be read. This function is responsible for accumulating the characters of the token and then calling eclector.reader:interpret-token in order to create and return a token.

Generic Function: interpret-token [eclector.reader] client input-stream token escape-ranges ¶

This generic function is called by eclector.reader:read-token in order to create a token from accumulated token characters. The parameter token is a string containing the characters that make up the token. The parameter escape-ranges indicates ranges of characters read from input-stream and preceded by a character with single-escape syntax or delimited by characters with multiple-escape syntax. Values of escape-ranges are lists of elements of the form (start . end) where start is the index of the first escaped character and end is the index following the last escaped character. Note that start and var can be identical indicating no escaped characters. This can happen in cases like a||b. The information conveyed by the escape-ranges parameter is used to convert the characters in token according to the readtable case of the current readtable before a token is constructed.

Generic Function: check-symbol-token [eclector.reader] client input-stream token escape-ranges position-package-marker-1 position-package-marker-2 ¶

This generic function is called by the default method on eclector.reader:interpret-token when the syntax of the token corresponds to that of a symbol. This function checks the syntactic validity of the symbol token and signals an error in case of a syntax error. If there are no syntax errors (or error recovery has been performed, see Recovering from errors), this function returns three values:

1. token or a value derived from token by error recovery operations.
2. position-package-marker-1 or a value derived from position-package-marker-1 by error recovery operations.
3. position-package-marker-2 or a value derived from position-package-marker-2 by error recovery operations.

The parameter input-stream is the input stream from which the characters were read. The parameter token is a string that contains all the characters of the token. The parameter escape-ranges indicates ranges within token that were preceded by a character with single-escape syntax or delimited by characters with multiple-escape syntax. The parameter position-package-marker-1 contains the index into token of the first package marker, or nil if the token contains no package markers. The parameter position-package-marker-2 contains the index into token of the second package marker, or nil if the token contains no package markers or only a single package marker.

The default method on this generic function checks the positions of the package markers taking into account escape ranges. The method signals errors and allows error recovery as described above.

Generic Function: interpret-symbol-token [eclector.reader] client input-stream token position-package-marker-1 position-package-marker-2 ¶

This generic function is called by the default method on eclector.reader:interpret-token when the syntax of the token corresponds to that of a valid symbol. The parameter input-stream is the input stream from which the characters were read. The parameter token is a string that contains all the characters of the token. The parameter position-package-marker-1 contains the index into token of the first package marker, or nil if the token contains no package markers. The parameter position-package-marker-2 contains the index into token of the second package marker, or nil if the token contains no package markers or only a single package marker.

The default method on this generic function calls eclector.reader:interpret-symbol with a symbol name string and a package indicator.

Generic Function: interpret-symbol [eclector.reader] client input-stream package-indicator symbol-name internp ¶

This generic function is called by the default method on eclector.reader:interpret-symbol-token as well as the default #: reader macro function to resolve a symbol name string and a package indicator to a representation of the designated symbol. The parameter input-stream is the input stream from which package-indicator and symbol-name were read. The parameter package-indicator is a either

• a string designating the package of that name
• the keyword :current designating the current package
• the keyword :keyword designating the keyword package
• nil to indicate that an uninterned symbol should be created

The symbol-name is the name of the desired symbol.

The default method uses cl:find-package (or *package* when package-indicator is :current) to resolve package-indicator followed by cl:find-symbol or cl:intern, depending on internp, to resolve symbol-name.

A second method which is specialized on package-indicator being nil uses cl:make-symbol to create uninterned symbols.

Generic Function: call-reader-macro [eclector.reader] client input-stream char readtable ¶

This generic function is called when the reader has determined that some character is associated with a reader macro. The parameter char has to be used in conjunction with the readtable parameter to obtain the macro function that is associated with the macro character. The parameter input-stream is the input stream from which the reader macro function will read additional input to accomplish its task.

The default method on this generic function simply obtains the reader macro function for char from readtable and calls it, passing input-stream and char as arguments. The default method therefore does the same thing that the standard Common Lisp reader does.

Generic Function: find-character [eclector.reader] client designator ¶

This generic function is called by the default #\ reader macro function to find a character. designator is either

• a string that is the name of the character to be found with single and multiple escapes removed, but with the case of all characters as it was in the input.
• or a character designating itself.

The function has to either return the character designated by designator or nil if no such character exists.

If designator is a string, it is the responsibility of the client to disregard the case of characters in designator, for example by producing an uppercase string from designator before looking up the designated character.

A default method on this generic function that is not specialized to any particular client but is specialized to designator being a string recognizes the mandatory character names listing in Section 13.1.7 Character Names of the Common Lisp specification. Another default method on this generic function that is not specialized to any particular client but is specialized to designator being a character just returns designator.

Generic Function: make-structure-instance [eclector.reader] client name initargs ¶

This generic function is called by the default #S reader macro function to construct structure instances. name is a symbol naming the structure type of which an instance should be constructed. initargs is a list the elements of which alternate between string designators naming structure slots and values for those slots.

It is the responsibility of the client to coerce the string designators to symbols as if by (intern (string slot-name) (find-package 'keyword)) as described in the Common Lisp specification.

There is no default method on this generic function since there is no portable way to construct structure instances given only the name of the structure type.

Generic Function: evaluate-expression [eclector.reader] client expression ¶

This generic function is called by the default #. reader macro function to perform read-time evaluation. expression is the expression that should be evaluated as it was returned by a recursive eclector.reader:read call and potentially influenced by client. The function has to either return the result of evaluating expression or signal an error.

The default method on this generic function simply returns the result of (cl:eval expression).

Generic Function: check-feature-expression [eclector.reader] client feature-expression ¶

This generic function is called by the default #+ and #- reader macro functions to check the well-formedness of feature-expression which has been read from the input stream before evaluating it. For compound expressions, only the outermost expression is checked regarding the atom in operator position and its shape – child expressions are not checked. The function returns an unspecified value if feature-expression is well-formed and signals an error otherwise.

The default method on this generic function accepts standard Common Lisp feature expressions, i.e. expressions recursively composed of symbols, :not-expressions, :and-expressions and :or-expressions.

Generic Function: evaluate-feature-expression [eclector.reader] client feature-expression ¶

This generic function is called by the default #+ and #- reader macro functions to evaluate feature-expression which has been read from the input stream. The function returns either true or false if feature-expression is well-formed and signals an error otherwise.

For compound feature expressions, the well-formedness of child expressions is not checked immediately but lazily, just before the child expression in question is evaluated in a subsequent evaluate-feature-expression call. This allows expressions like #+(and my-cl-implementation (special-feature a b)) form to be read without error when the :my-cl-implementation feature is absent.

The default method on this generic function first calls eclector.reader:check-feature-expression to check the well-formedness of feature-expression. It then evaluates feature-expression according to standard Common Lisp semantics for feature expressions.

2.3.3 Reader state protocol ¶

The reader state protocol consists of generic functions which the reader and the client call to query and modify the values of reader state aspects. Each aspect is named by a symbol and holds a current value and has a stack of shadowed values like a special variable. Most aspects roughly correspond to a particular reader control variable defined in the Common Lisp specification. In addition to those, Eclector uses aspects for representing the validity of the consing dot as well as the quasiquotation depth and validity in a given context. In total, Eclector defines the following aspects:

cl:*readtable*

Like the cl:*readtable* special variable, this aspect controls the readtable object in which the reader looks up the syntax types of characters, the case conversion mode as well as reader macros. By default, values of this aspect must satisfy the eclector.readtable:readtablep predicate.

cl:*package*

Like the cl:*package* special variable, this aspects controls the package which the reader uses when it looks up or interns symbols in the current package. By default, values of this aspect must be package designators.

cl:*read-suppress*

Like the cl:*read-suppress* special variable, this aspect controls whether the reader skips over expressions without detailed parsing.

cl:*read-eval*

Like the cl:*read-eval* special variable, this aspect controls whether the reader evaluates expressions in #. constructs.

cl:*features*

Like the cl:*features* special variable, this aspect controls the evaluation of features in feature expressions in #+ and #- constructs. By default, values of this aspect must be proper lists of symbols.

cl:*read-base*

Like the cl:*read-base* special variable, this aspect controls the interpretation of tokens by the reader as being integers or ratios. By default, values of this aspect must be of type (integer 1 36).

cl:*read-default-float-format*

Like the cl:*read-default-float-format* special variable, this aspect controls the floating-point format that the reader uses for floating-point numbers without exponent marker or the default exponent marker.

eclector.reader::*quasiquotation-state*

Warning: Clients should not query, bind or set the value of this aspect at this time.

This aspect controls whether backquote and unquote are allowed in the current context.

eclector.reader::*quasiquotation-depth*

Warning: Clients should not query, bind or set the value of this aspect at this time.

This aspect tracks the backquote nesting depth in the current context.

eclector.reader::*consing-dot-allowed-p*

Warning: Clients should not query, bind or set the value of this aspect at this time.

This aspect controls whether the consing dot is allowed in the current context.

Class: state-value-type-error [eclector.reader] ¶

Errors of this type are signaled when an attempt is made to establish an object as the value for a reader state aspect and the supplied object is not of the type required by the aspect.

Since this condition type is a subtype of cl:type-error, the offending value and the expected type can be retrieved via the readers cl:type-error-datum and cl:type-error-expected-type respectively. The aspect for which the value was supplied can be retrieved via the reader eclector.reader:aspect.

Generic Function: valid-state-value-p [eclector.reader] client aspect value ¶

This generic function is called by the reader to determine whether value is a valid value for the reader state aspect designated by aspect. The generic function returns true if, according to client, value is a valid value for the reader state aspect designated by aspect. aspect must designate a reader state aspect that is recognized by client. At least the aspects listed in the minimal reader state aspects table must be recognized by any client.

With the exceptions of cl:*readtable* and cl:*package*, the default methods on this generic function recognize state aspects and implement type restrictions informed by the Common Lisp specification:

Aspect	Type
cl:*readtable*	(satisfies eclector.readtable:readtablep)
cl:*package*	(or cl:package cl:symbol cl:string cl:character) (package designator)
cl:*read-suppress*	t (generalized Boolean)
cl:*read-eval*	t (generalized Boolean)
cl:*features*	list (proper list)
cl:*read-base*	(integer 2 36) (radix)
cl:*read-default-float-format*	(member short-float single-float double-float long-float)

Generic Function: state-value [eclector.reader] client aspect ¶

Return the current value of the reader state aspect designated by aspect.

aspect must designate a reader state aspect that is recognized by client. At least the aspects listed in the minimal reader state aspects table must be recognized by any client.

The cl:*package* aspect mandates further explanation: When the client uses only the default methods of the reader state protocol, the return value of this generic function for the cl:*package* aspect is of type cl:package which is a strict subtype of the type of valid values for this aspect. In other words, the defaults coerce package designators to package objects.

Generic Function: (setf state-value) [eclector.reader] new-value client aspect ¶

Set the current value of the reader state aspect designated by aspect to new-value.

aspect must designate a reader state aspect that is recognized by client. At least the aspects listed in the minimal reader state aspects table must be recognized by any client.

new-value is the desired new value for the designated aspect. new-value has to be a valid value for aspect in the sense that (eclector.reader:valid-state-value-p client aspect value) must return true.

The cl:*package* aspect mandates further explanation: When the client uses only the default methods of the reader state protocol, the method on this generic function which handles the cl:*package* aspect coerces new-value from designators to package objects so that a subsequent eclector.reader:state-value call returns the designated package object.

Generic Function: call-with-state-value [eclector.reader] client thunk aspect value ¶

Call thunk with the reader state aspect designated by aspect bound to value.

aspect must designate a reader state aspect that is recognized by client. At least the aspects listed in the minimal reader state aspects table must be recognized by any client.

The following properties must hold:

• value has to be a valid value for aspect in the sense that (eclector.reader:valid-state-value-p client aspect value) must return true.
• During the call to thunk and absent any intervening calls to eclector.reader:call-with-state-value, the expression (eclector.reader:state-value client aspect) must evaluate to value.

When Eclector calls this generic function with cl:*package* as the value of aspect, the value is a always a string designator and never a package object. The default method on this generic function coerces such string designators to package objects so that a subsequent eclector.reader:state-value call returns the designated package object.

Backquote and unquote syntax is forbidden in some contexts such as multi-dimensional array literals (#A) and structure literals (#S). Eclector tracks and controls whether backquote, unquote or both should be allowed in a given context using the aspects eclector.reader::*quasiquotation-state* and eclector.reader::*quasiquotation-depth* mentioned above. Since custom reader macros may also have to control this state, Eclector provides the following convenience macro:

Macro: with-quasiquotation-state [eclector.reader] client context quasiquote-forbidden-p unquote-forbidden-p &body body ¶

Warning: This macro is experimental and its name is not exported for now.

Control whether backquote syntax, unquote syntax or both are allowed in read functions called during the execution of body. context is a symbol identifying the current context which is used for error reporting. A typical value is the name of the reader macro function in which this macro is used. quasiquote-forbidden-p controls whether backquote syntax should be forbidden. The value :keep causes the binding to remain unchanged. unquote-forbidden-p controls whether unquote syntax should be forbidden. The value :keep causes the binding to remain unchanged.

Macro: with-forbidden-quasiquotation [eclector.reader] context &optional (quasiquote-forbidden-p t) (unquote-forbidden-p t) &body body ¶

Warning: This macro is deprecated as of Eclector 0.11 and will be removed in a future version. This macro is replaced by the macro eclector.reader:with-quasiquotation-state but that macro is experimental and its name is not exported for now.

Disallow backquote syntax, unquote syntax or both in read functions called during the execution of body. context is a symbol identifying the current context which is used for error reporting. A typical value is the name of the reader macro function in which this macro is used. quasiquote-forbidden-p controls whether backquote syntax should be forbidden. The value :keep causes the binding to remain unchanged. unquote-forbidden-p controls whether unquote syntax should be forbidden. The value :keep causes the binding to remain unchanged.

2.3.4 Labeled objects and references ¶

Eclector includes implementations of the #= and ## reader macros and they are present in the default readtable. One way to customize the behavior of the reader around the #= and ## syntax is replacing the reader macro functions with custom ones but with this approach the client code has to reimplement a lot of functionality. As a finer grained and more composable mechanism for customization, Eclector provides a protocol for implementing and customizing the behavior of the #= and ## reader macros, with or without modifying the readtable. The remainder of this section describes that protocol.

To start with a bit of terminology, we call the object created by reading #N=expression a labeled object. We call N the label of the labeled object and the result of reading expression the object of the labeled object. We say that #N=expression defines the labeled object and #N# references the labeled object. We call the reference circular if #N# occurs within expression. Labeled objects are internal to the reader and only exist during eclector.reader:read calls: before such a call returns an object, each labeled object within the returned object is replaced by its respective final object. Callers of eclector.reader:read and related functions will therefore only ever see the object, never the labeled object².

On a technical level, a labeled object is represented as a data type with a current state and a single (possibly unbound) slot containing the object. The following diagrams depicts the possible states of a labeled object together with input patterns and corresponding transitions:

Figure 2.3: Possible states of a labeled object and input patterns which correspond to state transitions.

Put differently, a labeled object can be in the following states:

The distinction between the states final and referenced on the one hand and final (circular), and referenced (circular) on the other hand is not required for implementing labeled objects. Those two pairs of states are therefore collapsed to just final and final (circular) in the remainder of this section. The following figure and paragraphs describe generic functions and methods which implement the creation, registration, lookup and manipulation of labeled objects according to the reduced set of states:

Figure 2.4: Reduced set of states of a labeled object and protocol functions with corresponding state transitions.

In addition to the generic functions referenced in the above diagram, the generic functions eclector.reader:fixup-graph-p, eclector.reader:fixup-graph, eclector.reader:fixup and eclector.reader:new-value-for-fixup are part of the protocol. Those functions are used to replace labeled objects with their respective final objects within an object that is about to be returned to the caller of eclector.reader:read ³. To this end, the #= reader macro function must inspect and update the state of the labeled object it is processing after reading expression by calling eclector.reader:finalize-labeled-object. eclector.reader:finalize-labeled-object decides whether eclector.reader:fixup-graph (see below) must be called: If after reading expression the labeled object is in state :circular, expression must have contained circular references and the result of reading it contains labeled objects that have to be replaced with their respective final objects. eclector.reader:fixup-graph, eclector.reader:fixup and eclector.reader:new-value-for-fixup perform this replacement. The replacement is performed by recursively⁴ traversing objects which are reachable from the final object of the labeled objects, for example by visiting the slots of standard objects, and replacing labeled objects with their respective final object.

In certain cases, the computational complexity of this traversal and replacement can be rather high, depending on when and how exactly the traversal is performed: consider an expression of the form #1=(1 #1# #2=(2 #2# …)). The nested labeled objects in this expression are all circular and thus require fixing up. The read call for the innermost labeled object, say #100=…, returns first and the fixup processing for the labeled object could be performed immediately. The problem is that each of the labeled objects would be processed in the same manner which would lead to a computation complexity of O(N M) where N is the number of labels and M is the number of nodes in the object graph rooted at the object which is returned by the outermost read call. One way to avoid this problem would be to perform fixup processing only for the outermost read call. The problem with that approach is that only a small sub-graph of the whole object graph may be circular in which case most of the work for traversing the whole graph would be wasted. To address both problems, Eclector allows clients to track the nesting of labeled objects and fix up sub-graphs which contain multiple nested objects in one go (see eclector.reader:fixup-graph-p).

Generic Function: call-with-label-tracking [eclector.reader] client thunk ¶

This generic function is called by the default method on eclector.reader:call-as-top-level-read in order to establish a context for tracking #= label definitions and ## label references around a call to thunk.

The default method on this generic function establishes a context in which the default #= and ## reader macro functions can make the appropriate calls to eclector.reader:note-labeled-object, eclector.reader:forget-labeled-object, eclector.reader:find-labeled-object.

Generic Function: note-labeled-object [eclector.reader] client input-stream label parent ¶

This generic function is called by the default #= reader macro function to note the definition of a labeled object with label label while reading from input-stream. The function creates, registers and returns a representation of the labeled object. The returned object is registered in the sense that a subsequent call to eclector.reader:find-labeled-object with arguments client and label returns the same object unless eclector.reader:forget-labeled-object has been called to unregister the object.

parent is either nil or a (previously created) surrounding labeled object. The parent labeled object is provided to allow the client to potentially defer fixup processing for the new labeled object if the processing for the surrounding labeled object subsumes the processing for the new labeled object.

Note that, when reading an expression of the form #N=object, this function is called after reading #N= from input-stream but before reading object. Consequently, the created and returned labeled object is defined but does not have an object associated with it.

The default method on this generic function calls eclector.reader:make-labeled-object with client, input-stream and label to create an object of an unspecified type. The method registers and returns the created object. Client code should manipulate the object only via the generic functions described in this section and in particular not rely on the object being of a particular type (since methods on eclector.reader:make-labeled-object specialized to certain client classes could return unexpected objects). The default method requires the context established by the default method on eclector.reader:call-with-label-tracking.

Generic Function: forget-labeled-object [eclector.reader] client label ¶

This generic function is called by the default #= reader macro function when Eclector reads an invalid labeled object of the form #N=#N# and the caller chooses to recover from the resulting error (see Recovering from errors). In that situation, the remainder of the input is processed as if there had been no labeled object with label N. This function makes the labeled object undefined so that a subsequent eclector.reader:find-labeled-object call for label will return nil.

The default method on this generic function requires the context established by the default method on eclector.reader:call-with-label-tracking.

Generic Function: find-labeled-object [eclector.reader] client label ¶

This generic function is called by the default ## reader macro function to look up the previously registered representation of a labeled object for label. The function returns nil if no such object has been registered for label and the registered object otherwise.

The default method on this generic function requires the context established by the default method on eclector.reader:call-with-label-tracking.

Generic Function: make-labeled-object [eclector.reader] client input-stream label parent ¶

This generic function is called by eclector.reader:note-labeled-object to create and return a representation of a labeled object with label label. parent is either nil or a previously created, surrounding labeled object which allows the client to potentially defer fixup processing for the new labeled object if the processing for the surrounding labeled object subsumes the processing.

The default method on this generic function creates and returns an object of an unspecified type. Client code should manipulate the object only via the generic functions eclector.reader:labeled-object-state, eclector.reader:finalize-labeled-object and eclector.reader:reference-labeled-object and in particular not rely on the object being of a particular type (since methods on this generic function specialized to certain client classes could return unexpected objects).

Generic Function: labeled-object-state [eclector.reader] client object ¶

This generic function is called by the default #= reader macro function to determine the state of object. This function returns

• nil if object is not a labeled object
• two values if object is a labeled object: one of the keywords :defined, :circular, :final, :final/circular and the final object stored in object if the first value is either :final or :final/circular or nil otherwise.

The following table lists all possible return value shapes:

object is a labeled object	First value	Second value
no	nil
yes	:defined	nil
yes	:circular	nil
yes	:final	final-object
yes	:final/circular	final-object

Note: This generic function may return more values in addition to the ones described above. Clients may use this feature to communicate additional information between methods (see Parse result construction features). Client defined methods on this generic function should accept such additional values when calling next methods and themselves return the additional values.

The default method on this generic function is applicable to labeled object representations returned by the default methods on eclector.reader:note-labeled-object and eclector.reader:make-labeled-object.

Generic Function: finalize-labeled-object [eclector.reader] client labeled-object object ¶

This generic function is called by the default #= reader macro function after reading a complete labeled object in order to store object in labeled-object and change the state of labeled-object to either :final or :final/circular. The function returns two values: the finalized labeled-object and the new state of labeled-object.

The default method on this generic function is applicable to labeled object representations returned by the default methods on eclector.reader:note-labeled-object and eclector.reader:make-labeled-object.

Generic Function: reference-labeled-object [eclector.reader] client input-stream labeled-object ¶

This generic function is called by the default ## reader macro function to process a reference to labeled-object while reading from input-stream. labeled-object must be a representation of a labeled object and has, in the context of the ## reader macro function, likely been obtained by calling eclector.reader:find-labeled-object. Depending on the state of labeled-object, this function returns either labeled-object itself or an object that can be returned to the caller as-is. In case labeled-object is returned, it will be replaced by its associated object later, when eclector.reader:fixup-graph is called.

The default method on this generic function is applicable to labeled object representations returned by the default methods on eclector.reader:note-labeled-object and eclector.reader:make-labeled-object.

As briefly mentioned above, the generic functions eclector.reader:fixup-graph and eclector.reader:fixup traverse and inspect objects in the object graph reachable from an object that is about to be returned to the caller of eclector.reader:read. In order to distinguish ordinary objects from labeled objects that act as placeholders in the object graph and must be replaced with their respective final objects, eclector.reader:fixup methods call eclector.reader:labeled-object-state on all encountered objects. eclector.reader:labeled-object-state returns nil for all objects that are not labeled objects and :final for labeled objects which must be replaced with their final object.

Generic Function: fixup-graph-p [eclector.reader] client root-labeled-object ¶

This generic function is potentially called by a method on eclector.reader:finalize-labeled-object to determine whether the object graph reachable from the object of root-labeled-object should be fixed up by calling eclector.reader:fixup-graph with client and labeled-object.

Multiple default methods on this generic function jointly implement the following behavior:

• If root-labeled-object has a parent labeled object, root-labeled-object should not be fixed up immediately (since the fixup processing for ancestor labeled objects will subsume the fixup processing for root-labeled-object).
• If root-labeled-object does not have a parent labeled object but has child labeled objects, root-labeled-object should be fixed up immediately.
• If root-labeled-object does not have a parent labeled object and is in state :final/circular, root-labeled-object should be fixed up immediately.

Generic Function: fixup-graph [eclector.reader] client root-labeled-object &key object-key ¶

This generic function is potentially called after the reader has constructed an object graph which is reachable from the object of root-labeled-object and noticed circular references within this graph to fix up circular references before the object of root-labeled-object is returned to the caller (of eclector.reader:read or related functions).

object-key is a function that accepts a labeled object and returns the object of the labeled object.

The default method on this generic function creates a hash table for tracking already processed objects and calls eclector.reader:fixup with client, the object of root-labeled-object and the hash table to recursively process objects in the object graph which is reachable from the object of root-labeled-object.

Generic Function: fixup [eclector.reader] client object traversal-state ¶

This generic function is potentially called to apply circularity-related changes to the object constructed by the reader before it is returned to the caller. object is the object that should be modified. traversal-state is an unspecified object that is used, among other things, to track already processed objects (see below). A method specialized to a class, instances of which consist of parts, should modify object by scanning its parts for labeled object markers, replacing found labeled object markers with the respective final object and recursively calling eclector.reader:fixup for all parts.

To recognize labeled objects which have to be replaced, methods should call eclector.reader:labeled-object-state on each part of object and interpret the returned values as follows: if nil is returned, the part should not be replaced but recursively processed. If :final is returned as the first value, the part should be replaced with the final object that is returned as the second value. Parts are replaced by mutating object.

This generic function is called for side effects – its return value is ignored.

Default methods specializing the object parameter to cl:cons, cl:array, cl:standard-object and cl:hash-table process instances of those classes by calling eclector.reader:new-value-for-fixup for each “place” in object and storing the returned object in the “place”.

An unspecialized :around method queries and updates traversal-state to ensure that each object reachable from object is processed exactly once. The method also limits the depth of nested function invocations (but importantly not the nesting depth or structure of the processed object graph, in fact the opposite: the limit on function invocation nesting is a consequence of allowing arbitrary object graphs to be processed).

Warning: Due to the limit on function call nesting, methods on this generic function must not rely on the traversal from the root object to object resulting in an uninterrupted chain of nested function calls. In particular, bindings of special variables and cl:unwind-protect cleanups that are established in a call which processes one object may not be in scope in calls which process objects that are logically at a deeper nesting level.

Generic Function: new-value-for-fixup [eclector.reader] client labeled-object current-value final-value ¶

This generic function is called by the generic function eclector.reader:fixup when a new value for a “place” within a circular object must be produced. This function is used when fixup processing is applied to both ordinary objects and parse results.

labeled-object is the labeled object marker which will be replaced by the computed value to finalize the circular object.

current-value is the current value of the “place” in the circular object. The value of this parameter is the same as labeled-object when fixup processing is applied to an ordinary object but is different when fixup processing is applied to a parse result. In the latter case, current-value is a parse result and labeled-object is the is ordinary object represented by the parse result.

final-value is the ordinary object that should replace labeled-object in the circular ordinary object regardless of whether fixup processing is performed on ordinary objects or parse results. In the latter case, methods on this generic function must return a suitable parse result that represents final-value and can replace current-value.

The default method on this generic function simply returns final-value.

A method specialized to (client cst-client) and (current-value cst:cst) returns a CST that represents final-value. Depending on the client, the returned CST is either another occurrence of the CST that represents the definition of final-value or a new CST that explicitly represents the labeled object reference.

2.3.5 S-expression creation ¶

The following generic functions allow clients to construct representations of quoted and quasiquoted forms as well as cl:function special forms.

Generic Function: wrap-in-quote [eclector.reader] client material ¶

This generic function is called by the default '-reader macro function to construct a quotation form in which material is the quoted material.

The default method on this generic function returns a result equivalent to (list 'common-lisp:quote material).

Generic Function: wrap-in-quasiquote [eclector.reader] client form ¶

This generic function is called by the default `-reader macro function to construct a quasiquotation form in which form is the quasiquoted material.

The default method on this generic function returns a result equivalent to (list 'eclector.reader:quasiquote form).

Generic Function: wrap-in-unquote [eclector.reader] client form ¶

This generic function is called by the default ,-reader macro function to construct an unquote form in which form is the unquoted material.

The default method on this generic function returns a result equivalent to (list 'eclector.reader:unquote form).

Generic Function: wrap-in-unquote-splicing [eclector.reader] client form ¶

This generic function is called by the default ,@-reader macro function to construct a splicing unquote form in which form is the unquoted material.

The default method on this generic function returns a result equivalent to (list 'eclector.reader:unquote-splicing form).

Generic Function: wrap-in-function [eclector.reader] client name ¶

This generic function is called by the default #'-reader macro function to construct a form that applies the function special operator to the name expression.

The default method on this generic function returns a result equivalent to (list 'common-lisp:function form).

2.3.6 Readtable initialization ¶

The standard syntax types and macro character associations used by the ordinary reader can be set up for any readtable object implementing the readtable protocol (see Readtable features). The following functions are provided for this purpose:

Function: set-standard-syntax-types [eclector.reader] readtable ¶

This function sets the standard syntax types in readtable (See Section 2.1.4 Character Syntax Types in the Common Lisp specification).

Function: set-standard-macro-characters [eclector.reader] readtable ¶

This function sets the standard macro characters in readtable (See Section 2.4 Standard Macro Characters in the Common Lisp specification).

Function: set-standard-dispatch-macro-characters [eclector.reader] readtable ¶

This function sets the standard dispatch macro characters, that is sharpsign and its sub-characters, in readtable (See Section 2.4.8 Sharpsign in the Common Lisp specification).

Function: set-standard-syntax-and-macros [eclector.reader] readtable ¶

This function sets the standard syntax types and macro characters in readtable by calling the above three functions.

4.1.1 Symbols and packages (default client) ¶

The default method on the generic function eclector.reader:interpret-symbol may create and intern symbols, thereby modifying the package system.

4.1.2 Read-time evaluation (default client) ¶

The default method on the generic function eclector.reader:evaluate-expression uses cl:eval to evaluate arbitrary expressions, potentially causing side effects. With the default readtable, the generic function is only called by the macro function of the #. reader macro.

4.1.3 Standard reader macros (default client) ¶

The default method on the generic function eclector.reader:call-reader-macro can cause side effects by calling macro functions that cause side effects. The following standard reader macros potentially cause side-effects:

• #. as described in Read-time evaluation (default client).

4.2.1 Symbols and packages ¶

In addition to the potential side effects described in Symbols and packages (default client), strings passed as the third argument of eclector.reader:interpret-token are potentially destructively modified during conversion to the current readtable case.

4.2.2 Read-time evaluation ¶

The same considerations as in Read-time evaluation (default client) apply.

4.2.3 Structure instance creation ¶

Clients defining methods on eclector.reader:make-structure-instance which implement the standard behavior of calling the default constructor (if any) of the named structure should consider side effects caused by slot initforms of the structure. The following example illustrates this problem:

  (defvar *counter* 0)
  (defstruct foo (bar (incf *counter*)))
  #S(foo)
  *counter* ⇒ 1
  #S(foo)
  *counter* ⇒ 2

4.2.4 Circular structure ¶

The eclector.reader:fixup generic function potentially modifies its second argument destructively. Clients that define methods on eclector.reader:make-structure-instance should be aware of this potential modification in cases like #1=#S(foo :bar #1#). Similar considerations apply for other ways of constructing compound objects such as #1=(t . #1#).

4.2.5 Standard reader macros ¶

The following standard reader macros could cause or be affected by side effects when combined with a non-standard client:

• #. as described in Read-time evaluation (default client).
• #S as described in Structure instance creation.
• (, #( and #S as described in Circular structure.
• The ,. (i.e. destructively splicing) variant of the , reader macro does not currently destructively modify the surrounding object, but clients should not rely on this fact. This consideration applies to clients that install non-standard macro functions for the ( and #( reader macros.