PSP Development/Lua

Lua Analysis

This article suggests you have an adept understanding of Lua outside. This isn't a Lua syntax article.

Replacing C with Lua

It must be stated that porting everything from C to Lua is basically wrapping everything with complexity. This means creating a function in C, which links to a Lua function identifier somewhere - Perhaps in an organized table. While it is certainly easy to do, you have to understand Lua from the inside, not just the outside. LuaC can become daunting trying to monitor the stack.

Objectified Lua

While Lua adds bulk to C, great ways to utilize it are AI, GUI, and controls. Lua is a very powerful language, and with the ability to do JIT compilation, flexibility is a major plus. Unlike replacing C with Lua, having a more objectified reason to use Lua is preferable.

Manually Building Difficulty

Lua doesn't compile out of the box with anything but normal PCs. However, you can build Lua5.3.4 successfully and run it. It just takes a little bit of manual work. It may be difficult.

• psp-gcc should be used, which isn't used by the premade Makefile (just standard gcc), alongside the psp-gcc includes and libs
• It is mandatory that you use -DLUA_USE_C89 and -DLUA_C89_NUMBERS for each of the files. The psp-gcc compiler doesn't use long long. C89 is good to use because the age of the PSP and our target version of PSP firmware 1.5
• The headers need to match to install it, or use it as a static library. lua.h, lauxlib.h, lualib.h, and luaconf.h (included in lua.h) Modification of luaconf.h necessary.
• When mass-compiling all C files to O files, there will be lua.o and luac.o, which contain main methods and will throw redefinition errors, and need to stay out of a static library
• The correct libm.a is needed by the PSP, which is different than Windows or Linux

Getting Lua

Lua is a free software and library available under a MIT license at lua.org. In order to build, and use it, understanding the license is necessary. Lua has many years under its belt, thusly it is built to support the most basic compilers. A few tweaks and the build should go smoothly.

Downloading

Lua5.1.5 build for the PSP: (click here)

Lua5.2.4 build for the PSP: (click here)

Lua5.3.4 build for the PSP: (click here)

Building

Preparing/Compiling

1. Download a PSP source package from lua.org
2. Extract only the src folder
3. Delete the Makefile inside, as it won't be used
4. (If applicable) In luaconf.h, uncomment #define LUA_32BITS and #define LUA_USE_C89
5. Open a terminal or cmd in that directory

Run this code:

psp-gcc -DLUA_USE_C89 -DLUA_32BITS -DLUA_C89_NUMBERS -O2 -G0 -Wall -c *.c

Cleaning/Organizing

1. Delete lua.o and luac.o object files
2. Copy all .o files to a new, empty directory
3. Copy the header files lua.h, lua.hpp (depends on version), luaconf.h, and lauxlib.h into a new directory
4. Enter the directory where your .o files are
5. open a terminal or cmd in that directory.

Archiving

Run this code:

ar -cvq liblua.a *.o

1. Copy liblua.a where you temporarily stored the header files in #3
2. Delete the new .o files directory created in #2

Installing

1. Place liblua.a inside of psp/sdk/lib folder
2. Place all .h files inside of psp/sdk/include folder

Setting Up Makefile

The libraries in use are libm.a and liblua.a, which was installed into psp/sdk/include and psp/sdk/lib. The LIBS variable in the Makefile houses libraries to be used.

1. Add -llua under LIBS
2. Add -lm under LIBS

TARGET = LUATEST
OBJS = main.o

INCDIR =
CFLAGS = -O2 -G0 -Wall
CXXFLAGS = $(CFLAGS) -fno-exceptions -fno-rtti
ASFLAGS = $(CFLAGS)

LIBDIR =
LIBS = -llua -lm
LDFLAGS =

EXTRA_TARGETS = EBOOT.PBP
PSP_EBOOT_TITLE = Lua test program

PSPSDK=$(shell psp-config --pspsdk-path)
include $(PSPSDK)/lib/build.mak

Using Lua in C

This is the heart of what you will be doing. A lot of this code that we will do can be considered abstract, in that you can create a compiler directive to create LuaC functions in one swift call. However, that is beyond the scope of this article at the time of writing.

Includes

First you will need includes. Always remember the includes.

main.c

// Standard libraries for PSP
#include <pspkernel.h>
#include <pspdebug.h>
#include <pspdisplay.h>

// Lua includes
#include "lua.h"
#include "lualib.h"
#include "lauxlib.h"

// Macro keyword printf to point to pspDebugScreenPrintf for ease
#define printf pspDebugScreenPrintf

// Setup the module and main thread
#define VERS 1
#define REVS 0
PSP_MODULE_INFO("LUATEST", PSP_MODULE_USER, VERS, REVS);
PSP_MAIN_THREAD_ATTR(PSP_THREAD_ATTR_USER);

Creating a Common Lua Space

Now you have all the Lua functions in your immediate top-level function space. In the package provided, there was also lua.hpp, which is a C++ header. You will have to look through it if you want to use it C++-esque, which it should be very similar.

Now we should define a print. Printing is so necessary. It is your number 1 debugging tool. We need to create a function, which is in C, but is put into Lua's global scope to call from inside Lua. It is better to not let it clutter the main.c compilation unit, and since you can make changes to either the classes for adjustment, you can save on compilation time.

lua_lib.c

#include "lua_lib.h"

// Lua includes
#include "lua.h"
#include "lualib.h"
#include "lauxlib.h"

lua_lib.h

#ifndef LUA_LIB_H
#define LUA_LIB_H

// Lua includes
#include "lua.h"
#include "lualib.h"
#include "lauxlib.h"

#endif

Now that we have an empty compilation unit, we need to add it to our Makefile OBJS.

OBJS = main.o lua_lib.o

Now that we have everything set up, you can build and watch as an empty object file is created next to lua_lib.c. However, now its time to fill it!

Creating the Lua Context

Everything balances on a lua_State pointer. Think of this as your global space. When you create a new state, you create a very new 'context'. That is, THERE ARE NO FUNCTIONS! __VERSION, os.*, etc is NOT inside this state - its naked. We need to give it life. So let's create such a function that will give us such a state. However, we are going to call luaL_openlibs(L) to give us all the standard libraries, not go with just the naked globals.

lua_lib.h

lua_State* lua_create_newstate();

lua_lib.c

lua_State* lua_create_newstate()
{
	lua_State* L;
	L = luaL_newstate();
	luaL_openlibs(L);

	return L;
}

Adding Functions to Lua

Okay out next step would be the almighty print function! To do this, we are going to use an extern function, which returns an int. It is important to return an int because it is used by Lua. Lua has a very dynamic return system. You should prefix anything that has to immediately do with lua with lua_. The reason for extern goes back to what we are doing immediately. The extern has little to do with much outside of assembly, a great replacement is static, but it is extern for correctness. You need to have an understanding of what extern does to get it. The int returned is the number of arguments the function returns. Finally, when Lua calls the C function to use it, it will give it the lua_State pointer you created. MAKE SURE YOU INCLUDE THE PROPER HEADERS. You can basically slap in everything you included in main into lua_lib.c.

lua_lib.c

extern int lua_print(lua_State *L)
{
	int arg_len = lua_gettop(L);
	int n;
	for (n=1; n <= arg_len; n++) {
		printf("%s%s", lua_tostring(L, n), "\t");
	}
	printf("\n");
	return 0;
}

Now to explain the meat of the function. Lua is a stack based scripting language. A good practice would be to error on an invalid amount of arguments passed. However, since print is basically a tuple, we can ignore this. Calling lua_gettop(L) will give you the number of things pushed on the stack. lua_tostring(L, n) returns a static char* of your argument from a lua string. A careful practice, but being stubborn, would be to luaL_checkstring(L, n), which does the same exact thing, but errors if you don't give it a string. In Lua, you should not be so stubborn to allow only strings because tables, functions, userdata, etc are all printable, in-fact tostring should return something like function: pointer.

This specific function prints everything with a tab between it. It is really, really helpful for when you need to debug multiple parameters or unpack(a table). Don't forget to end the line though, as you will get easily confused. One call to print in Lua is one output line with the given arguments displayed, spanning right indefinitely - as in doesn't wrap.

Moving on, we need to register this function. The print() function is in global space. This means we can just register the function. It is easiest to do this in the lua_create_newstate() function, as that is where we build the lua_State already. Remember that the lua_State is where we have our function environment. We use the function lua_register() to do such a task. The first argument is the lua_State, the second is the name of the function. (Remember this is going in global space) The final parameter is the reference to our function we created.

lua_lib.c

lua_State* lua_create_newstate()
{
	lua_State* L;
	L = luaL_newstate();
	luaL_openlibs(L);
	
	lua_register(L, "print", lua_print);
	
	return L;
}

Great! Now getfenv()["print"] now will invoke the function lua_print, letting C manage the stack. Now we have to go over to the main compilation unit and set everything up. Lua for PSP is pretty picky and you must know what you are doing and understand why it is being done.

Creating the Test Lua Script

We should create the script that we will run. Create a script called script.lua in the root project directory. This file should be in the same directory as your EBOOT.PBP. We are going to create a simple script that will run once per loop in the main loop. This allows us to draw our prints with pspDebugScreenPrintf, and not have them erased on next draw iteration.

script.lua

-- print a simple lua_string
print("Hello from Lua!")

Putting it All Together

Now let's go over to the main.

We should do the basic callback setup and initialize the debug screen. This allows us to draw our text. Please note the parameters in our main function.

main.c

int main(int argc, char** argv)
{
	setupExitCallback();
	pspDebugScreenInit();

Let's talk about the parameters and paths for int main(). When you launch a typical EBOOT.PBP on PPSSPP, you will get umd0:/EBOOT.PBP. We are stationed at umd0:/. This is everything in your EBOOT.PBP folder, which is in your PSP's Memstick > /PSP/GAME/EBOOT_FOLDER. To be safe, you could dynamically strip the EBOOT.PBP off the protocol. However, until you face the problem, this solution works! That would require some string manipulation.

Next we need to load in the file. To begin doing this, we first need to grab the path. We also need to initialize a new state and load the libraries. Luckily for us, we have the lua_lib_newstate() we created to do this for us.

	
	// script file must be in the same directory as EBOOT.PBP
	const str* scriptPath = "umd0:/script.lua"; // wherever your script is located
	
	// init Lua and load all libraries
	lua_State *L = lua_lib_newstate();
	
	int status = 0;

DO NOT USE THE CODE BELOW. While we aren't supposed to exit the program, we set the debug screen back to position. We have a few choices. First, we can call luaL_dofile to load then execute the file. We will be reading from storage pretty fast. This isn't the behavior that we want. This is also the equivalent of calling luaL_loadfile then lua_pcall(). What we want to do is load it in once, cache that load, and use it from there. Alongside this, if you have 4 scripts that are called per iteration, you load that data from storage 4 times. At 60 fps, that is 240 read accesses to storage. DO NOT USE THE CODE BELOW.

	// call script
	// DO NOT RUN THIS. IT ACCESSES FILES VERY, VERY FAST. PROTECT YOUR SSD/MEMSTICK LIFESPAN.
	while (isRunning()) {
		pspDebugScreenSetXY(0, 0);
		status = luaL_dofile(L, scriptPath);
		printf("test");
		//status = lua_pcall(L, 0, 2, 0);
		sceDisplayWaitVblankStart();
	}

To circumvent the problem above, we need to do the following. First, what you want to do is load the file. After, use luaL_ref to pop the loaded file check from the stack (prevents the execution), and make a hard reference to the index it gives you. Your file chunk is cached as that index in the LUA_REGISTRY(INDEX). After, you should rawget it back from registry to run. That way you are reading from memory instead of umd0:/ constantly. It is cached. pcall it and everything should work! Make sure you do error checks, which you will discover later in the page. If you load 4 scripts this way, you would be reading, jumping around the memory instead from the storage. You would only need to access the storage medium 4 times initially.

	// init Lua, its libraries, and do a dry run for initialization
	lua_State *L = lua_lib_newstate();
	
	int status = 0;
	
	// cache the file in lua registry
	status = luaL_loadfile(L, scriptPath);
	
	// TODO: Error check the status for compilation error
	
	int lua_script = luaL_ref(L, LUA_REGISTRYINDEX);

Now you can use this code below, unlike the example earlier, to hook this all up. It is mandatory for you to check for errors, please read on to learn how to manage your stack and keep it clean.

	// run main logic, call script, check for errors
	while (isRunning()) {
		pspDebugScreenSetXY(0, 0);
		lua_rawgeti(L, LUA_REGISTRYINDEX, lua_script); // get script into top of stack
		status = lua_pcall(L, 0, 0, 0); // run the script
		// TODO: check for runtime errors
		sceDisplayWaitVblankStart();
	}

Now all that is left is cleaning up the program. This is where you free your lua_State/stack, etc. If you crash the program, you do not need to wipe the stack yourself, but rely on lua_close. Then do your basic exit game and return. You want to make sure you clean up any dynamic memory here that you used with Lua. I HIGHLY recommend NOT dynamically allocating things lua-side or c-side that you can't immediately clean up before returning a const.

	// cleanup
	lua_close(L);
	
	sceKernelExitGame();	
	return 0;
}

Congratulations! You should now be able to run a lua script! All you need to do is call make EBOOT.PBP and resolve anything you may have mis-typed, such as things in the Makefile, main.c, or lua_lib.c/.h.

Error Checking/Stack Management

This is how you should error check. Error checking is a must if you are developing scripts, especially if they are dynamic from run to run. It is also important to have a clean stack.

It is important to know what is on the stack. You will push functions, strings, numbers, etc (The latter are from returns). If Lua doesn't handle them, you WILL HAVE TO pop them from the stack. Let's say you only have one function that you call in lua after loading a script up to it. It returns a string. Your stack is now dirty because the return from the function isn't handled. When a script dies, you need to clean up. This is where this handy stackDump function comes in. Use it.

main.c

static void stackDump(lua_State *L) {
	int i = lua_gettop(L);
	printf("---------------- Stack Dump ----------------\n");
	while(i) {
		int t = lua_type(L, i);
		switch (t) {
		case LUA_TSTRING:
			printf("%d:`%s`\n", i, lua_tostring(L, i));
			break;
		case LUA_TBOOLEAN:
			printf("%d: %s\n", i, lua_toboolean(L, i) ? "true" : "false");
			break;
		case LUA_TNUMBER:
			printf("%d: %g\n", i, lua_tonumber(L, i));
			break;
		default:
			printf("%d: %s\n", i, lua_typename(L, t));
			break;
		}
		i--;
	}
	printf("--------------- Stack Dump Finished ---------------\n");
}

Now let's implement it! The following code is the loop seen in the previous sections above, but with stackDump. If you were to run this with a Lua script that doesn't error, you will find the stack is always clean after you call lua_pcall, unless the code returned values. This isn't allowed, which generates an error, but can be cut around by do return end. The stack could be clean in that instance. Between lua_rawgeti and lua_pcall below, you will have things pushed on your stack, which lua_pcall will digest (the top of stack will be a function, as that is what your loadfile puts on the stack - your script wrapped in a function). You really can't stress how much a clean stack is vital. You risk undefined behavior and memory issues. The PSP isn't some monstrous beast computer with 16 gigs of WRAM, 4 gigs of VRAM, and a CPU clocked at 4+ gigahertz that can take a bit of a beating.

main.c

	// run main logic, call script, check for errors
	while (isRunning()) {
		pspDebugScreenSetXY(0, 0);
		lua_rawgeti(L, LUA_REGISTRYINDEX, lua_script); // get script into stack
		status = lua_pcall(L, 0, 0, 0); // run the script
		if(status != 0) {
			// alerting that we have a runtime error
			stackDump(L); // a dirty stack with error string
			printf(lua_tostring(L, -1)); // print last push, ie the error
			lua_pop(L, 1); // pop the error arg
			stackDump(L); // a clean stack
		}
		sceDisplayWaitVblankStart();
	}

Working Example

• main.c : http://pastebin.com/VxTJaAXp
• lua_lib.c : http://pastebin.com/iJRpC4cP
• lua_lib.h : http://pastebin.com/NUwVZwUT