Introduction to newLISP/More examples
More examples
[edit | edit source]This section contains some simple examples of newLISP in action. You can find plenty of good newLISP code on the web and in the standard newLISP distribution.
On your own terms
[edit | edit source]You might find that you don't like the names of some of the newLISP functions. You can use constant and global to assign another symbol to the function:
(constant (global 'set!) setf)
You can now use set! instead of setf. There's no speed penalty for doing this.
It's also possible to define your own alternatives to built-in functions. For example, earlier we defined a context and a default function that did the same job as println, but kept a count of how many characters were output. For this code to be evaluated rather than the built-in code, do the following.
First, define the function:
(define (Output:Output)
(if Output:counter
(inc Output:counter (length (string (args))))
(set 'Output:counter 0))
(map print (args))
(print "\n"))
Keep the original newLISP version of println available by defining an alias for it:
(constant (global 'newLISP-println) println)
Assign the println symbol to your Output function:
(constant (global 'println) Output)
Now you can use println as usual:
(for (i 1 10)
(println (inc i)))
2 3 4 5 6 7 8 9 10 11
(map println '(1 2 3 4 5))
1 2 3 4 5
And it appears to do the same job as the original function. But now you can also make use of the bonus features of the alternative println that you defined:
Output:counter
;-> 36
; or
println:counter
;-> 36
In case you've been counting carefully - the counter has been counting the length of the arguments supplied to the Output function. These include the parentheses, of course...
Using a SQLite database
[edit | edit source]Sometimes it's easier to use existing software rather than write all the routines yourself, even though it can be fun designing something from the beginning. For example, you can save much time and effort by using an existing database engine such as SQLite instead of building custom data structures and database access functions. Here's how you might use the SQLite database engine with newLISP.
Suppose you have a set of data that you want to analyse. For example, I've found a list of information about the elements in the periodic table, stored as a simple space-delimited table:
(set 'elements
[text]1 1.0079 Hydrogen H -259 -253 0.09 0.14 1776 1 13.5984
2 4.0026 Helium He -272 -269 0 0 1895 18 24.5874
3 6.941 Lithium Li 180 1347 0.53 0 1817 1 5.3917
...
108 277 Hassium Hs 0 0 0 0 1984 8 0
109 268 Meitnerium Mt 0 0 0 0 1982 9 0[/text])
(You can find the list in this file on GitHub..)
The columns here are Atomic Weight, Melting Point, Boiling Point, Density, Percentage in the earth's crust, Year of discovery, Group, and Ionization energy. (I used 0 to mean Not Applicable, which was not a very good choice, as it turns out).
To load newLISP's SQLite module, use the following line:
(load "/usr/share/newlisp/modules/sqlite3.lsp")
This loads in the newLISP source file which contains the SQLite interface. It also creates a new context called sql3, containing the functions and symbols for working with SQLite databases.
Next, we want to create a new database or open an existing one:
(if (sql3:open "periodic_table")
(println "database opened/created")
(println "problem: " (sql3:error)))
This creates a new SQLite database file called periodic_table, and opens it. If the file already exists, it will be opened ready for use. You don't have to refer to this database again, because newLISP's SQLite library routines maintain a current database in the sql3 context. If the open function fails, the most recent error stored in sql3:error is printed instead.
I've just created this database, so the next step is to create a table. First, though, I'll define a symbol containing a string of column names and the SQLite data types each one should use. I don't have to do this, but it probably has to be written down somewhere, so, instead of the back of an envelope, write it in a newLISP symbol:
(set 'column-def "number INTEGER, atomic_weight FLOAT,
element TEXT, symbol TEXT, mp FLOAT, bp FLOAT, density
FLOAT, earth_crust FLOAT, discovered INTEGER, egroup
INTEGER, ionization FLOAT")
Now I can make a function that creates the table:
(define (create-table)
(if (sql3:sql (string "create table t1 (" column-def ")"))
(println "created table ... OK")
(println "problem " (sql3:error))))
It's easy because I've just created the column-def symbol in exactly the right format! This function uses the sql3:sql function to create a table called t1.
I want one more function: one that fills the SQLite table with the data stored in the list elements. It's not a pretty function, but it does the job, and it only has to be called once.
(define (init-table)
(dolist (e (parse elements "\n" 0))
(set 'line (parse e))
(if (sql3:sql
(format "insert into t1 values (%d,%f,'%s','%s',%f,%f,%f,%f,%d,%d,%f);"
(int (line 0))
(float (line 1))
(line 2)
(line 3)
(float (line 4))
(float (line 5))
(float (line 6))
(float (line 7))
(int (line 8))
(int (line 9))
(float (line 10))))
; success
(println "inserted element " e)
; failure
(println (sql3:error) ":" "problem inserting " e))))
This function calls parse twice. The first parse breaks the data into lines. The second parse breaks up each of these lines into a list of fields. Then I can use format to enclose the value of each field in single quotes, remembering to change the strings to integers or floating-point numbers (using int and float) according to the column definitions.
It's now time to build the database:
(if (not (find "t1" (sql3:tables)))
(and
(create-table)
(init-table)))
- if the t1 table doesn't exist in a list of tables, the functions that create and fill it are called.
Querying the data
[edit | edit source]The database is now ready for use. But first I'll write a simple utility function to make queries easier:
(define (query sql-text)
(set 'sqlarray (sql3:sql sql-text)) ; results of query
(if sqlarray
(map println sqlarray)
(println (sql3:error) " query problem ")))
This function submits the supplied text, and either prints out the results (by mapping println over the results list) or displays the error message.
Here are a few sample queries.
Find all elements discovered before 1900 that make up more than 2% of the earth's crust, and display the results sorted by their discovery date:
(query
"select element,earth_crust,discovered
from t1
where discovered < 1900 and earth_crust > 2
order by discovered")
("Iron" 5.05 0)
("Magnesium" 2.08 1755)
("Oxygen" 46.71 1774)
("Potassium" 2.58 1807)
("Sodium" 2.75 1807)
("Calcium" 3.65 1808)
("Silicon" 27.69 1824)
("Aluminium" 8.07 1825)
When were the noble gases discovered (they are in group 18)?
(query
"select symbol, element, discovered
from t1
where egroup = 18")
("He" "Helium" 1895)
("Ne" "Neon" 1898)
("Ar" "Argon" 1894)
("Kr" "Krypton" 1898)
("Xe" "Xenon" 1898)
("Rn" "Radon" 1900)
What are the atomic weights of all elements whose symbols start with A?
(query
"select element,symbol,atomic_weight
from t1
where symbol like 'A%'
order by element")
("Actinium" "Ac" 227)
("Aluminium" "Al" 26.9815)
("Americium" "Am" 243)
("Argon" "Ar" 39.948)
("Arsenic" "As" 74.9216)
("Astatine" "At" 210)
("Gold" "Au" 196.9665)
("Silver" "Ag" 107.8682)
It's elementary, my dear Watson! Perhaps the scientists out there can supply some examples of more scientifically interesting queries?
You can find MySQL and Postgres modules for newLISP on the net as well.
Simple countdown timer
[edit | edit source]Next is a simple countdown timer that runs as a command-line utility. This example shows some techniques for accessing the command-line arguments in a script.
To start a countdown, you type the command (the name of the newLISP script) followed by a duration. The duration can be in seconds; minutes and seconds; hours, minutes, and seconds; or even days, hours, minutes, and seconds, separated by colons. It can also be any newLISP expression.
> countdown 30 Started countdown of 00d 00h 00m 30s at 2006-09-05 15:44:17 Finish time: 2006-09-05 15:44:47 Elapsed: 00d 00h 00m 11s Remaining: 00d 00h 00m 19s
or:
> countdown 1:30 Started countdown of 00d 00h 01m 30s at 2006-09-05 15:44:47 Finish time: 2006-09-05 15:46:17 Elapsed: 00d 00h 00m 02s Remaining: 00d 00h 01m 28s
or:
> countdown 1:00:00 Started countdown of 00d 01h 00m 00s at 2006-09-05 15:45:15 Finish time: 2006-09-05 16:45:15 Elapsed: 00d 00h 00m 02s Remaining: 00d 00h 59m 58s
or:
> countdown 5:04:00:00 Started countdown of 05d 04h 00m 00s at 2006-09-05 15:45:47 Finish time: 2006-09-10 19:45:47 Elapsed: 00d 00h 00m 05s Remaining: 05d 03h 59m 55s
Alternatively, you can supply a newLISP expression instead of a numerical duration. This might be a simple calculation, such as the number of seconds in π minutes:
> countdown "(mul 60 (mul 2 (acos 0)))" Started countdown of 00d 00h 03m 08s at 2006-09-05 15:52:49 Finish time: 2006-09-05 15:55:57 Elapsed: 00d 00h 00m 08s Remaining: 00d 00h 03m 00s
or, more usefully, a countdown to a specific moment in time, which you supply by subtracting the time now from the time of the target:
> countdown "(- (date-value 2006 12 25) (date-value))" Started countdown of 110d 08h 50m 50s at 2006-09-05 16:09:10 Finish time: 2006-12-25 00:00:00 Elapsed: 00d 00h 00m 07s Remaining: 110d 08h 50m 43s
- in this example we've specified Christmas Day using date-value, which returns the number of seconds since 1970 for specified dates and times.
The evaluation of expressions is done by eval-string, and here it's applied to the input text if it starts with "(" - generally a clue that there's a newLISP expression around! Otherwise the input is assumed to be colon-delimited, and is split up by parse and converted into seconds.
The information is taken from the arguments given on the command line, and extracted using main-args, which is a list of the arguments that were used when the program was run:
(main-args 2)
This fetches argument 2; argument 0 is the name of the newLISP program, argument 1 is the name of the script, so argument 2 is the first string following the countdown command.
Save this file as countdown, and make it executable.
#!/usr/bin/newlisp
(if (not (main-args 2))
(begin
(println "usage: countdown duration [message]\n
specify duration in seconds or d:h:m:s")
(exit)))
(define (set-duration)
; convert input to seconds
(if (starts-with duration-input "(")
(set 'duration-input (string (eval-string duration-input))))
(set 'duration
(dolist (e (reverse (parse duration-input ":")))
(if (!= e)
(inc duration (mul (int e) ('(1 60 3600 86400) $idx)))))))
(define (seconds->dhms s)
; convert seconds to day hour min sec display
(letn
((secs (mod s 60))
(mins (mod (div s 60) 60))
(hours (mod (div s 3600) 24))
(days (mod (div s 86400) 86400)))
(format "%02dd %02dh %02dm %02ds" days hours mins secs)))
(define (clear-screen-normans-way)
; clear screen using codes - thanks to norman on newlisp forum :-)
(println "\027[H\027[2J"))
(define (notify announcement)
; MacOS X-only code. Change for other platforms.
(and
(= ostype "OSX")
; beep thrice
(exec (string {osascript -e 'tell application "Finder" to beep 3'}))
; speak announcment:
(if (!= announcement nil)
(exec (string {osascript -e 'say "} announcement {"'})))
; notify using Growl:
(exec (format
"/usr/local/bin/growlnotify %s -m \"Finished count down \""
(date (date-value) 0 "%Y-%m-%d %H:%M:%S")))))
(set 'duration-input (main-args 2) 'duration 0)
(set-duration)
(set 'start-time (date-value))
(set 'target-time (add (date-value) duration))
(set 'banner
(string "Started countdown of "
(seconds->dhms duration)
" at "
(date start-time 0 "%Y-%m-%d %H:%M:%S")
"\nFinish time: "
(date target-time 0 "%Y-%m-%d %H:%M:%S")))
(while (<= (date-value) target-time)
(clear-screen-normans-way)
(println
banner
"\n\n"
"Elapsed: "
(seconds->dhms (- (date-value) start-time ))
" Remaining: "
(seconds->dhms (abs (- (date-value) target-time))))
(sleep 1000))
(println
"Countdown completed at "
(date (date-value) 0
"%Y-%m-%d %H:%M:%S") "\n")
; do any notifications here
(notify (main-args 3))
(exit)
Editing text files in folders and hierarchies
[edit | edit source]Here's a simple function that updates some text date stamps in every file in a folder, by looking for enclosing tags and changing the text between them. For example, you might have a pair of tags holding the date the file was last edited, such as <last-edited> and </last-edited>.
(define (replace-string-in-files start-str end-str repl-str folder)
(set 'path (real-path folder))
(set 'file-list (directory folder {^[^.]}))
(dolist (f file-list)
(println "processing file " f)
(set 'the-file (string path "/" f))
(set 'page (read-file the-file))
(replace
(append start-str "(.*?)" end-str) ; pattern
page ; text
(append start-str repl-str end-str) ; replacement
0) ; regex option number
(write-file the-file page)
))
which can be called like this:
(replace-string-in-files
{<last-edited>} {</last-edited>}
(date (date-value) 0 "%Y-%m-%d %H:%M:%S")
"/Users/me/Desktop/temp/")
The replace-string-in-files function accepts a folder name. The first task is to extract a list of suitable files - we're using directory with the regular expression {^[^.]} to exclude all files that start with a dot. Then, for each file, the contents are loaded into a symbol, the replace function replaces text enclosed in the specified strings, and finally the modified text is saved back to disk. To call the function, specify the start and end tags, followed by the text and the folder name. In this example we're just using a simple ISO date stamp provided by date and date-value.
Recursive version
[edit | edit source]Suppose we now wanted to make this work for folders within folders within folders, ie to traverse a hierarchy of files, changing every file on the way down. To do this, re-factor the replace-string function so that it works on a passed pathname. Then write a recursive function to look for folders within folders, and generate all the required pathnames, passing each one to the replace-string function. This re-factoring might be a good thing to do anyway: it makes the first function simpler, for one thing.
(define (replace-string-in-file start-str end-str repl-str pn)
(println "processing file " pn)
(set 'page (read-file pn))
(replace
(append start-str "(.*?)" end-str) ; pattern
page ; text
(append start-str repl-str end-str) ; replacement
0) ; regex option number
(write-file pn page))
Next for that recursive tree-walking function. This looks at each normal entry in a folder/directory, and tests to see if it's a directory (using directory?). If it is, the replace-in-tree function calls itself and starts again at the new location. If it isn't, the pathname of the file is passed to the replace-string-in-file function.
(define (replace-in-tree dir s e r)
(dolist (nde (directory dir {^[^.]}))
(if (directory? (append dir nde))
(replace-in-tree (append dir nde "/") s e r)
(replace-string-in-file (append dir nde) s e r))))
To change a tree-full of files, call the function like this:
(replace-in-tree
{/Users/me/Desktop/temp/}
{<last-edited>}
{</last-edited>}
(date (date-value) 0 "%Y-%m-%d %H:%M:%S"))
It's important to test these things in a scratch area first; a small mistake in your code could make a big impact on your data. Caveat newLISPer!
Talking to other applications (MacOS X example)
[edit | edit source]newLISP provides a good environment for gluing together features found in application programs that have their own scripting languages. It's fast and small enough to keep out of the way of the other components of a scripted solution, and it's good for processing information as it passes through your workflow.
Here is an example of how you can use a newLISP script to send non-newLISP scripting commands to applications. The task is to construct a circle in Adobe Illustrator, given three points on the circumference.
The solution is in three parts. First, we obtain the coordinates of the selection from the application. Next we calculate the radius and centre point of the circle that passes through these points. Finally, we can draw the circle. The first and final parts use AppleScript, which is run using the osascript command, because Adobe Illustrator doesn't understand any other scripting language (on Windows, you use Visual Basic rather than AppleScript).
The calculations and general interfacing are carried out using newLISP. This can often be a better solution than using native AppleScript, because newLISP offers many powerful string and mathematical functions that can't be found in the default AppleScript system. For example, if I want to use trigonometry I would have to find and install an extra component - AppleScript doesn't provide any trig functions at all.
The newLISP script can sit in the Scripts menu on the menu bar; put it into the Library > Scripts > Applications > Adobe Illustrator folder, which accepts both text files and AppleScripts). It's then ready to be selected while you're working in Illustrator. To use it, just select a path with at least three points, then run the script. The first three points define the location for the new circle.
#!/usr/bin/newlisp
; geometry routines from
; http://cgafaq.info/wiki/Circle_Through_Three_Points
; given three points, draw a circle through them
(set 'pointslist
(exec
(format [text]osascript -e 'tell application "Adobe Illustrator 10"
tell front document
set s to selection
repeat with p in s
set firstItem to p
set pathinfo to entire path of firstItem
set pointslist to ""
repeat with p1 in pathinfo
set a to anchor of p1
set pointslist to pointslist & " " & item 1 of a
set pointslist to pointslist & " " & item 2 of a
end repeat
end repeat
end tell
end tell
pointslist'
[/text])))
; cleanup
(set 'points
(filter float?
(map float (parse (first pointslist) { } 0))))
(set 'ax (points 0)
'ay (points 1)
'bx (points 2)
'by (points 3)
'cx (points 4)
'cy (points 5))
(set 'A (sub bx ax)
'B (sub by ay)
'C (sub cx ax)
'D (sub cy ay)
'E (add
(mul A (add ax bx))
(mul B (add ay by)))
'F (add
(mul C (add ax cx))
(mul D (add ay cy)))
'G (mul 2
(sub
(mul A (sub cy by))
(mul B (sub cx bx)))))
(if (= G 0) ; collinear, forget it
(exit))
(set 'centre-x (div (sub (mul D E) (mul B F)) G)
'centre-y (div (sub (mul A F) (mul C E)) G)
'r
(sqrt
(add
(pow (sub ax centre-x))
(pow (sub ay centre-y)))))
; we have coords of centre and the radius
; in centre-x, centre-y, and r
; Illustrator bounds are left-x, top-y, right-x, bottom-y
; ie centre-x - r, centre-y + r, centre-x + r, centre-y -r
(set 'bounds-string
(string "{" (sub centre-x r) ", "
(add centre-y r) ", "
(add centre-x r) ", "
(sub centre-y r) "}"))
(set 'draw-circle
(exec (format [text]osascript -e 'tell application "Adobe Illustrator 10"
tell front document
set e to make new ellipse at beginning with properties {bounds:%s}
end tell
end tell
'
[/text] bounds-string)))
(exit)
There's hardly any error handling in this script! More should definitely be added to the first stage (because the selection might not be suitable for subsequent processing).