Lush/Compiling

Lush is an interpreted language, but individual functions and class methods may be compiled, either to just speed it up, or to enable inline C/C++ code.

Compiling pure lush functions

Functions and class methods may be compiled, with a few modifications. Here's a simple function that we'd like to compile:

 ;; Prints the size, sum, and mean of a vector
 (de print-mean (vec)
   (let* ((v-size (idx-dim vec 0))
          (v-sum (idx-sum vec)
          (v-mean (/ v-sum v-size))
     (printf "Sum = %f, size = %d, mean = %f\n" v-sum v-size v-mean)
     ())

Below we've added some type declarations make the function compilable, and followed it with a compilation command:

 ;; Prints the size, sum, and mean of a vector
 (de print-mean (vec)
   ((-idx1- (-double-)) vec)
   (let* ((v-size (idx-dim vec 0))
          (v-sum (idx-sum vec)
          (v-mean (/ v-sum v-size))
     ((-int-) v-size)
     (printf "Sum = %f, size = %d, mean = %f\n" v-sum v-size v-mean)
     ())
 
 ; compile "print-mean"
 (dhc-make () print-mean)

To be compilable, lush functions must:

• Declare the types of any objects of ambiguous type.
• Use only compilable functions and expressions.
• List the function in the compilation statement at the end of the source file.

Declare types

In the above function, the lines immediately following the arguments declare the argument types. Note the function doesn't bother declaring the types of any other variable. This is because you only need to declare types for function arguments and non-double numeric types. This is because function argument types aren't evident at compile-time unless specified, and lush assumes that all numbers are doubles unless told otherwise.

All type declarations are of the form:

((type_name) var_name)

For example:

((-int-) my-int)

The following lists the type names of the compilable lush types, along with their C++ equivalents:

Lush type	C++ type	Description
-int-	int	32-bit integer
-float-	float	32-bit real
-double-	double	64-bit real
-short-	short	16-bit integer
-byte-	char	8-bit integer
-ubyte-	unsigned char	8-bit unsigned integer
-gptr-	void*	Type-indpenedent pointer (size is platform dependent).
-gptr- "std::string"	std::string*	Type-specific pointer (same size as -gptr-/void*)
-str-	?	Lush string
-idx- (-double-)	?	Lush tensor (-double- could be any basic type name.*)
-obj- (htable)	?	Any other class (htable is the class name for Lush hash tables).

* Tensors can't contain type-specific pointers. In other words, the following is not allowed:

;; NOT ALLOWED: a tensor of char* pointers
((-idx1- (-gptr- "char")) my-char-pointer-tensor)

But this would be fine:

;; A tensor of void* pointers
((-idx2- (-gptr-)) my-pointer-tensor)

Only use compilable functions

Unfortunately, not all lush functions or expressions are compilable. Most functions that are not compilable are those that involve list manipulation. More generally, any particularly "lispy" feature of lush (lambda forms, hooks, code manipulation) will usually not be compilable. To see whether a particular function is compilable, use the function compilablep:

 ? (compilablep double-matrix)
 = t
 ? (compilablep range)
 = ()

Note that compilablep will only take single function and macro names, not complex lush expressions.

While list manipulation is generally non-compilable, an expression with a list in it is not necessarily non-compilable. For example, the function idx-transclone necessarily takes a list as one of its arguments, but is still compilable.

Add a compilation command

After making your functions compilation-friendly, you must call a compilation function at the end of the source file to actually compile them. Here's a function and two class methods, followed by the compilation function dhc-make:

 (de some-func (...)
   ... )
 
 (de my-class object
   ... )
 
 (defmethod my-class method-1 (...)
   ... )
 
 (defmethod my-class method-2 (...)
   ... )
 
 (dhc-make ()
           (some-func)
           (my-class method-1
                     method-2))

The compilation function actually does 3 things:

• Transcribes the functions/methods into equivalent C code. The C source files for the file src-dir/src-file.lsh will be in src-dir/C/src_file.c.
• Compiles this C source file into an object file using gcc.
• Links the resulting object file into memory.

The compilation function only runs if the C source file is older than the Lush source file.

Mixing in C/C++ code

You can insert C/C++ code into lush functions using the #{ ... #} construct. You can interleave lush and native code pretty densely:

Accessing lush data from C/C++ code

The table below shows how to access different lush objects from C/C++ code:

Type	lush name	C/C++ name
basic type	my-double	$my_double
tensor type	my-float-tensor	float* raw_data = IDX_PTR( $my_float_tensor, float )
string	my-str	char* c_string = $my_str->data
object slots	:my-obj:some-slot	$my_obj->some_slot
object method	(==> my-obj some-method arg1)	$my_obj->vtbl->M_some_method($arg1)
global function	(add-numbers num1 num2)	C_add_numbers($num1, $num2)
global constant	@MY_CONSTANT	MY_CONSTANT

C code in the compilation function

You can stick preprocessor statements and other C code right into the compilation function:

C functions that call the Lush functions

If there are lots of them, you can put them in .h and .c files for better readability:

 (dhc-make ()
           #{
           #include "c_funcs.h"
           #}
           
           lush-func-1
           lush-func-2
           ''...''
           
           #{
           #include "c_funcs.c"
           #})

Linking to libraries

libload can return null; be careful.

Compiling and linking C/C++ source files

lushmake. Proper lushmake syntax? The way I do it, it doesn't pay attention to dependencies.

Quirks

The problem of hidden arguments

One problem with lush functions compiled into C code is that any lush objects that you instantiate within the function become arguments in the C version of the function. These arguments that only show up in the C version of a lush function are called "hidden arguments." the lush function's signature (e.g. the number and types of arguments it takes) can change. This is because of

No void return type

Lush functions that return nil (a.k.a. ()) will return a char in its C incarnation. This can be a problem when you really need your function to not return anything, say when using it as a callback function for some C API that expects a void-returning function:

A nil-returning function in my-src.lsh:

 (de my-callback ()
   ...
   ())

...becomes a char-returning function in C/my_src.c

 char* C_my_callback(){
 ...
 }

In this case, your best bet is to wrap the lush function in a C function that returns void. The question then is: how do I compile this wrapper function? The simplest way is to stick it right in the compilation function at the end of my-src.lsh:

 (dhc-make ()
           #{ void callback_wrapper(); #}
           my-callback
           ''... other lush functions & methods ...''
           ''...''
           ''...''
           #{ void callback_wrapper(){ C_my_callback(); } #} )

Stack or Heap?:

No cout

Dynamic allocation of wrapped C++ classes: the casting bug

Issues to consider before compiling

There are several issues to consider before deciding whether or not you really want to compile your lush code.

to be compiled can be more of a hassle than writing the same code in c/c++, for a number of reasons:

• Compilable Lush isn't as feature-rich or visually clean as C++
• Lush compiler errors can be even more opaque than those of C/C++ compilers, especially if macros are involved.
• Loss of debuggability