Slimv Tutorial - Part Two

Sometimes variables or function calls may be optimized away, so it's rather complicated to examine the backtrace. In this case we may add the following declaim form at the top of the source file, this instructs lisp to produce the safest code and show all debug information:

(declaim (optimize (speed 0) (safety 3) (debug 3)))

At this point in his movie Marco presents some Paredit trickery. We skip this topic for now, it will be covered in another part of the Slimv tutorial. Instead we go on adding some new functions, that are going to transform strings into morse sequences and back.

The first one is string-to-morse. Here Marco demonstrates Slime's hierarchical compiler notes buffer by compiling a buggy version of the function. Unfortunately there is no compiler notes buffer in Slimv at the moment, rather all errors and warning messages are printed in the REPL buffer together with all compilation information supplied by the Swank server. Let's see an example of a problematic loop form:

(defun string-to-morse (string)
  (with-output-to-string (morse)
    (write-string (character-to-morse (aref string 0)) morse)
    (loop
      do
      for char across (subseq string 1)
      do (write-char #\Space morse)
      do (write-string (character-to-morse char) morse))))

And this is the compilation result of string-to-morse:

MORSE> 
; compiling file "/home/kovisoft/morse.lisp" (written 30 APR 2011 05:00:32 PM):

; file: /home/kovisoft/morse.lisp
; in: DEFUN STRING-TO-MORSE
;     (LOOP DO
;           MORSE::FOR CHAR MORSE::ACROSS (SUBSEQ STRING 1)
;           DO (WRITE-CHAR #\  MORSE::MORSE)
;           DO (WRITE-STRING (MORSE::CHARACTER-TO-MORSE CHAR) MORSE::MORSE))
; 
; caught ERROR:
;   (in macroexpansion of (LOOP DO FOR ...))
;   (hint: For more precise location, try *BREAK-ON-SIGNALS*.)
;   A compound form was expected, but FOR found.
;   current LOOP context: DO FOR CHAR.

;     (WITH-OUTPUT-TO-STRING (MORSE::MORSE)
;       (WRITE-STRING (MORSE::CHARACTER-TO-MORSE (AREF STRING 0)) MORSE::MORSE)
;       (LOOP DO
;             MORSE::FOR CHAR MORSE::ACROSS (SUBSEQ STRING 1)
;             DO (WRITE-CHAR #\  MORSE::MORSE)
;             DO (WRITE-STRING (MORSE::CHARACTER-TO-MORSE CHAR) MORSE::MORSE)))
; --> LET GET-OUTPUT-STREAM-STRING 
; ==>
;   MORSE::MORSE
; 
; note: deleting unreachable code
; 
; compilation unit finished
;   caught 1 ERROR condition
;   printed 1 note

; /home/kovisoft/morse.fasl written
; compilation finished in 0:00:00.090

2 compiler notes:

/home/kovisoft/morse.lisp:1219
  error: (in macroexpansion of (LOOP DO FOR ...))
(hint: For more precise location, try *BREAK-ON-SIGNALS*.)
A compound form was expected, but FOR found.
current LOOP context: DO FOR CHAR.

/home/kovisoft/morse.lisp:1120
  note: deleting unreachable code

This long list informs us that there is a problem in the macroexpansion of (LOOP DO FOR ...). We can collect information about the proper usage of the loop form in various ways.

One way is to hover the mouse over the loop symbol: this will display a tooltip with the symbol description. Another way to describe a symbol is to place the cursor on the symbol name and press ,s (or select Documentation/Describe-Symbol from the Slimv menu):

COMMON-LISP:LOOP
  [symbol]

LOOP names a macro:
  Lambda-list: (&ENVIRONMENT ENV &REST KEYWORDS-AND-FORMS)
  Source file: SYS:SRC;CODE;LOOP.LISP
Press ENTER or type command to continue 

We can also look up the symbol in the Common Lisp Hyperspec by pressing ,h (or selecting Documentation/Hyperspec from the Slimv menu). This will open the related Hyperspec page in the default browser.

But as we know that loop is a macro, therefore we can also examine its macroexpansion by pressing ,1 on the form (or selecting Debugging/Macroexpand-1 from the Slimv menu):

A compound form was expected, but FOR found.
current LOOP context: DO FOR CHAR.
   [Condition of type SB-INT:SIMPLE-PROGRAM-ERROR]

Restarts:
  0: [*ABORT] Return to SLIME's top level.
  1: [TERMINATE-THREAD] Terminate this thread (#<THREAD "worker" RUNNING {B2C63D1}>)

Backtrace:
  0: (SB-LOOP::LOOP-ERROR "A compound form was expected, but ~S found.")[:EXTERNAL]
  1: (SB-LOOP::LOOP-GET-COMPOUND-FORM)
  2: (SB-LOOP::LOOP-GET-PROGN)
  3: (SB-LOOP::LOOP-DO-DO)
  4: (SB-LOOP::LOOP-ITERATION-DRIVER)
  5: (SB-LOOP::LOOP-TRANSLATE (DO FOR CHAR ACROSS (SUBSEQ STRING 1) ...
  6: (MACROEXPAND-1 ..)
Slimv.SLDB [RO]^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^5,1^^^^^^^^^^^^Top
     (loop                                                                               
       do                                                                                
       for char across (subseq string 1)                                                 
                                                                                         
morse.lisp                                                            58,6            Bot

Now we see that the problem is the extra do before for. Select the 0: [*ABORT] restart and eliminate the extra line:

(defun string-to-morse (string)
  (with-output-to-string (morse)
    (write-string (character-to-morse (aref string 0)) morse)
    (loop
      for char across (subseq string 1)
      do (write-char #\Space morse)
      do (write-string (character-to-morse char) morse))))

After macroexpanding the loop form again (make sure that the cursor stays inside the the loop form, but not inside an inner sub-form) we get something like that:

MORSE> 
;; Macroexpand-1 was selected here on (loop ...)
(BLOCK NIL
  (LET ((CHAR NIL)
        (#:LOOP-ACROSS-VECTOR-862 (SUBSEQ STRING 1))
        (#:LOOP-ACROSS-INDEX-863 0)
        (#:LOOP-ACROSS-LIMIT-864 0))
    (DECLARE (TYPE FIXNUM #:LOOP-ACROSS-LIMIT-864)
             (TYPE FIXNUM #:LOOP-ACROSS-INDEX-863)
             (TYPE VECTOR #:LOOP-ACROSS-VECTOR-862))
    (SB-LOOP::LOOP-BODY
     ((SETQ #:LOOP-ACROSS-LIMIT-864 (LENGTH #:LOOP-ACROSS-VECTOR-862)))
     ((WHEN (>= #:LOOP-ACROSS-INDEX-863 #:LOOP-ACROSS-LIMIT-864)
        (GO SB-LOOP::END-LOOP))
      (SB-LOOP::LOOP-REALLY-DESETQ CHAR
                                   (AREF #:LOOP-ACROSS-VECTOR-862
                                         #:LOOP-ACROSS-INDEX-863))
      NIL
      (SB-LOOP::LOOP-REALLY-DESETQ #:LOOP-ACROSS-INDEX-863
                                   (1+ #:LOOP-ACROSS-INDEX-863)))
     ((WRITE-CHAR #\  MORSE) (WRITE-STRING (CHARACTER-TO-MORSE CHAR) MORSE))
     ((WHEN (>= #:LOOP-ACROSS-INDEX-863 #:LOOP-ACROSS-LIMIT-864)
        (GO SB-LOOP::END-LOOP))
      (SB-LOOP::LOOP-REALLY-DESETQ CHAR
                                   (AREF #:LOOP-ACROSS-VECTOR-862
                                         #:LOOP-ACROSS-INDEX-863))
      NIL
      (SB-LOOP::LOOP-REALLY-DESETQ #:LOOP-ACROSS-INDEX-863
                                   (1+ #:LOOP-ACROSS-INDEX-863)))
     NIL)))

Right now we don't decrypt the output, but at least we can confirm the macroexpansion does not give any errors. Let's compile the corrected function and test it in the REPL buffer:

MORSE> (string-to-morse "marco")
"-- .- .-. -.-. ---"
MORSE> (string-to-morse "sos")
"... --- ..."

Let's verify that the newly installed package is available via find-package. Inspect the package by pressing ,i (or by selecting Debugging/Inspect from the Slimv menu). We get a prompt to enter the symbol name to inspect. The prompt might be pre-filled with a symbol name under the cursor, but we ignore this and use * (i.e. the result of the previous find-package expression - the package object is this case).

The inspector has a hierarchical structure. You can inspect the n-th part below the current node by pressing Enter in Normal mode on a line beginning with [nn], where nn is the part identifier. You can return to the previous node by pressing Enter on the last line containing [<<].
There may also be "actions" bound to the node, these are marked with <nn>, where nn is the action identifier. You also select actions by pressing Enter on their line.

If we are new to a package, we can inspect the external symbols provided by the package. Then we can examine the function descriptions one by one:

CL-USER> (find-package :split-sequence)
#<PACKAGE "SPLIT-SEQUENCE">
CL-USER> 
;; "Inspect: *" was used here


Slimv.REPL.lisp                                                      112,0-1          All
Inspecting #<PACKAGE {B99A001}>
--------------------
[1] Name: "SPLIT-SEQUENCE"
[2] Nick names: "PARTITION"
[3] Use list: COMMON-LISP
Used by list: 
[4] 35 present symbols.
[5] 6 external symbols.
[6] 29 internal symbols.
[7] 978 inherited symbols.
0 shadowed symbols.
 
[<<]
;; Item [5] was selected here
Slimv.INSPECT [RO]^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^8,1^^^^^^^^^^^^Top
Inspecting #<SWANK::%PACKAGE-SYMBOLS-CONTAINER {AF5EC21}>
--------------------
All external symbols of package "SPLIT-SEQUENCE"

A symbol is considered external of a package if it's
"part of the `external interface' to the package and
[is] inherited by any other package that uses the
package." (CLHS glossary entry of `external')

<0>   [Group by classification]

Symbols:                       Flags:
------------------------------ --------
[1] PARTITION                  -f-----p
[2] PARTITION-IF               -f------
[3] PARTITION-IF-NOT           -f------
[4] SPLIT-SEQUENCE             -f-----p
[5] SPLIT-SEQUENCE-IF          -f------
[6] SPLIT-SEQUENCE-IF-NOT      -f------
 
[<<]
;; Item [4] was selected here
Slimv.INSPECT [RO]^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^17,1^^^^^^^^^^^^Top
Inspecting #<SYMBOL {B99A1AF}>
--------------------
[1] Its name is: "SPLIT-SEQUENCE"
It is unbound.
[2] It is a function: #<FUNCTION SPLIT-SEQUENCE:SPLIT-SEQUENCE>
<0> [unbind]
Function documentation:
  Return a list of subsequences in seq delimited by delimiter.

If :remove-empty-subseqs is NIL, empty subsequences will be included
in the result; otherwise they will be discarded.  All other keywords
work analogously to those for CL:SUBSTITUTE.  In particular, the
behaviour of :from-end is possibly different from other versions of
this function; :from-end values of NIL and T are equivalent unless
:count is supplied. The second return value is an index suitable as an
argument to CL:SUBSEQ into the sequence indicating where processing
stopped.

[3] It is external to the package: SPLIT-SEQUENCE
<1> [unintern]
[4] Property list: NIL
[5] It names the package: @0=#<PACKAGE "SPLIT-SEQUENCE">
 
[<<]
Slimv.INSPECT [RO]^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^1,1^^^^^^^^^^^^Top

In the above example we selected the same items in the inspector as Marco did in the movie. Our actions are marked by green lines, these are explanatory messages, they are normally not printed by the REPL.

Now package split-sequence is installed and ready for use in morse-to-string.