Civ/Civilization IV/Modding/Tutorials/Python Tutorial

Introduction

Python can be used to achieve a surprising amount of things in Civ 4, but only if you know how. Now, while I'm probably not the best person to be writing this tutorial, I feel I am quite familiar with the sort of things you can do to the game with it. Please feel free to edit anything I've got here, as the tutorial is far from complete. The tutorial is also on CFC here. Any updates on the wiki I will attempt to update there, though obviously the format is a bit stricter there.

If you want to know a little bit about python, I'd check out the wikipedia entry and the python website

Just a brief rundown on what python is capable of in Civ 4 terms:

Python can:

be used to make complex scripts causing events to happen on a trigger. For example, you can make it so that if you take the holy city of a religion, all Civs following that religion will declare war on you.
be used to edit the interface. Pretty much all of the game interface is generated dynamically by python, and can be edited in python. You can make new buttons, alter the layout of the Civelopedia, change the way the tech tree is generated... and even, if you so desire make a small dancing pig in the corner of your screen - with music (though I don't believe anybody has tried this, I'm pretty sure it's doable). There are limitation to this - mouseover texts, for example, are far better suited to the SDK.
be used to generate new map types. I haven't tried this myself, so it might not be covered too well in this tutorial.
be used to edit the AI... I'd do it with the SDK though, as it'll probably save a lot of time!

Python can't:

be used exactly how you might want to use it. While the amount of things you can do in it is really quite large, there are several things you just can't do. This is really quite annoying, but it's something you've got to learn to live with. The amount of things you can do is staggering, so don't complain. Hopefully with the release of the SDK more experienced coders will enhance the ability of python to change the way the game works.

Still reading? Good!

Now my suggestion for first steps would be to browse through GBM's python tutorial, which gives a brief outline of how to program in python. Also, if you have not done so, check out the python website which has lots of information about python (unsuprisingly). Finally, I'd check out this python tutorial by Jon (Trip) Shafer, Firaxis, who's tutorial you may find better than mine, after all, he is the guy who coded a lot of it in the first place.

The basics

As you should know by now, there are several different types of "thing" you can get in python: ints, floats, strings, and bools. You also might be aware of "pointers". I wasn't when I first came to python, so I'll give you a little heads-up about them.

Basically a pointer is a game entity. It can take the form of a unit, a plot (square), or a player. In the default python code, and in many modder's code, a variable is shown to be a pointer by prefacing it with "p". So pUnit will define a single unit in the game, pPlayer a single player.

Likewise, if a variable is prefaced with an "i" it is likely to be an integer (int) variable, and "b" a Boolean (bool) variable. iPlayer, for example, will probably be a player's individual ID number, rather then the pointer for the player.

It does sometimes get confusing, as most pointers have integers which are closely related to them. This method of naming things helps to avoid confusion in a language like python where variables need not be declared, however it is only a guideline, and just because a variable is named as if it were a certain type of variable, it won't necessarily be one.

I also would like to put a note here about "Types". In most xml entries you will need to enter something into a type field. This field is a very important link to python, though must be converted to an int for it to work properly. There is one function that you'll be finding yourself using over and over again, so I thought I'd put it here. This function will find the int corresponding to the type - and as most functions in the API (soon to come) will want the int value of the type it's very useful. For example:

gc.getInfoTypeForString("TECH_MYSTICISM")

will return the int corresponding to the tech Mysticism, which, as it is the first entry in the relevant xml file will be zero by default.

gc.getInfoTypeForString("RELIGION_TAOISM")

will return the int 6 by default, as it is the 7th religion listed.

If you don't get that last bit right now, don't worry - hopefully it'll come clear when you read the next section.

The API

The Civ 4 API is a list of all the functions specific to Civ 4 which interface with the actual game. There are two copies of the API. The first, by Locutus can be found here, and the second, by GBM can be found here. Locutus' API is up to date with the patch 1.60, so I suggest you use that one.

Now, in the past month or so answering questions on the forums about python I have noticed that quite a few people have had trouble reading the APIs. Interpreting what the functions do can be quite difficult, however the APIs do give you lots of hints, if you care to read them.

I'll use Locutus' API as an example, as I feel the interface is better, so open it up!

Classes

Firstly, we have the classes. These list all the functions that can be done to a specific entity, or pointer. For example, CyUnit lists all the functions that will work on a unit entity (a unit in the game). The classes are listed in the top-left hand frame. You will notice that there are several classes with a little "+" symbol by them. These classes are used for getting specific information directly from the xml files on an object.

Types

Now we have the types. These are the different types which can be extracted from the xml files. The int values shown are the default values, and while you could use them, and your code would most likely work, I would recommend against it, as if the order of the xml is changed in a future patch, your code will break. Instead I recommend using gc.getInfoTypeForString("") as shown above, as this is much more patch-compatible.

Functions

Finally we have the functions. Each class has its own set of functions which must be done to that class and that class only. If you try to take a function from a class, and use it on a different class it won't work. The functions are the main bit of the API, the stuff you want to know.

Now, there are lots of different functions. Most of the functions don't "do" anything in the game, instead they get a value from your pointer which you can use in an equation. If you look on the left hand side of the function, it will show you what this function returns. For example, if you used:

BOOL CyPlayer.canChangeReligion()

it would not set the player able to change religion, instead it would return a BOOL, a true or false value depending on weather the player can or can't. See what I mean about it not "doing" anything?

Things a function can return are:

Bool - true or false, 1 or 0
Int - an integer value (can be negative) (256 for example)
Float - a floating point number (1.3423 for example)
String - a string ("The cat sat on the mat")
Turple - a list ([1, 4, 2, 8, 2])
Void - see below

The functions that do "do" things return a "VOID" - meaning that they setting rather than getting data. These functions are the most important ones really, as you can't do much without them, and unfortunately there aren't really enough of them to do everything you would want. Sorry, but that's just the way it is.

So, you know what a function returns, but that's not all you need to know. You also need to know what information to give it to return this data. Some functions don't require any inputs.

CyUnit.canMove()

for example doesn't need to know anything apart from which unit it's checking to see if can move (the unit pointer should be in the place of the CyUnit). However, most functions aren't this simple, and require an input to create an output.

Let's take, for example:

CyUnit.setHasPromotion(PromotionType eIndex, BOOL bNewValue)

This function takes two arguments (inputs), the promotion type you want to give to the unit, and a bool value of weather to take, or remove the promotion. To give an example:

pUnit.setHasPromotion(gc.getInfoTypeForString("PROMOTION_COMBAT1"), 1)

would give the pointer, pUnit the promotion combat 1. Note that it is important to give the int value of the promotion in this case, as when the function asks for a type, it really wants an int... don't ask! As luck would have it, if you put the wrong arguments into a function python will tell you when trying to run the function. More about this in the debugging section!

CyGlobalContext

Commonly abbreviated to gc in files this is probably the most useful class, as it contains all the general Civ 4 related functions. GlobalContext is unique in that it doesn't have to be called with a pointer, but is a standalone class for getting information.

It has a number of useful purposes:

As I have said before you can use it to get integers from types. For example:

gc.getInfoTypeForString("TECH_MYSTICISM")

would get the int value for Mysticism.

You can use it to get from an integer ID to a pointer. For example:

gc.getPlayer(0)

would return the pPlayer pointer for the the player with an ID of zero.

You can use it to get info from xml files. For example

gc.getPromotionInfo("PROMOTION_COMBAT1")

would return the pointer for the combat 1 promotion, which could then be used with CvPromotionInfo to get information about that specific promotion.

A common usage is to get the active player pointer, which returns the pointer of the player whose turn it is:

gc.getActivePlayer()

The final major use is to get the number of instances of a certain thing. For example:

gc.getNumPromotionInfos()

would return the number of promotions which are available in the game. This is very useful for if you want to cycle through the promotions and check if a unit has them, activating a new function if it does.

While there are other uses for CyGlobalContext, they are a bit to numerous to list here. I've tried to list what I think are the major ones.

Events

Events are things which occur due to certain triggers. There is only certain amount of fixed triggers, and this ties down modders slightly, however the triggers will accommodate most scripts. If you open up CvEventManager.py, the event triggers run from line 193 down to line 747 (patch 1.52). There are nearly 60 different triggers that can be used, ranging from onUpdate which runs every frame (probably about 30 times a second), to onGameStart which runs only once, at the start of the game.

Most triggers have arguments associated with them. These arguments are stored in the argsList. Under most event headers these arguments are defined. It's usually pretty obvious what they do.

Perhaps the simplest method of adding events on triggers is to just to add the event into the events manager. This will work for small mods, although it raises certain compatibility issues. A better way of doing it is to create your own CvCustomEventManager file. I recommend Dr Elmer Jiggle's event manager (found here). It's a bit tricky to understand at first, but well worth it if you can get it working. For examples of it in action I'd check out any of TheLopez's ModComps (look in this forum).

Things to note

The onEndPlayerTurn tigger does not occur at the end of a player's turn, instead it occurs at the end of the start of a player's turn, after all the cities have built, etc. If you want to do an event on the end of a player's turn you'll probably have to bodge up some code to do it at the start of the next player's turn. It would appear something similar goes on with onBeginGameTurn, and onEndGameTurn, although both occur at the END of the game turn rather than at the beginning like the player ones. Thanks to Kael for spotting this.

Example

If, for instance, you wanted a message to come up on every player's screen at the start of every game turn (in other words, at the start of the first player's turn), you could replace this code in the event manager:

def onBeginGameTurn(self, argsList):
	'Called at the beginning of the end of each turn'
	iGameTurn = argsList[0]
	CvTopCivs.CvTopCivs().turnChecker(iGameTurn)

with the following code:

def onBeginGameTurn(self, argsList):
	'Called at the beginning of the end of each turn'
	iGameTurn = argsList[0]
	CyInterface.addImmediateMessage("You have just started a new turn", "") # Adds the message "You have just started a new turn" with no sound attached.
 	CvTopCivs.CvTopCivs().turnChecker(iGameTurn)

NOTE: When making a formal mod you shouldn't simply add stuff to the event manager like this as it can cause compatibility issues. Check out the custom event manager linked to above.

Interface

As I mentioned in the introduction almost all of the GUI is created dynamically either when you enter the relevant screen, or at the start of the game. Just about all of it is moddable. The files for modding the interface are in .../Assets/Python/Screens.

Things to note

When placing items in the interface that the x and y co-ords that you enter will correspond to the top left hand corner of the item.

Remember that the resolution can change. It may often be a good idea to change how your interface displays depending on the resolution. To find the X and Y resolutions you can use the code:

CyGInterfaceScreen.getXResolution() or CyGInterfaceScreen.getYResolution()

If an item needs a name, that name must be unique to that item, or strange things start to happen.

The way the techchooser script is written can make it quite hard to mod. Instead of being generated everytime you load up the screen, the techchooser is instead generated at the start of the game, and then only the colour changes are done in-game. To stop this happening in game, and to ensure that the techchooser shows the mods you make to it while you're game is running I suggest you comment out line 72 (screen.setPersistent(True)). This will make the techchooser be constantly regenerated while open, and although this may cause your computer to slow slightly, it should work better. Sometimes though, due to the way it is generated for techchooser mods you will have to restart the game to see any changes.

Debugging

I would appreciate it if somebody could check over this section as I'm not certain as to all of it's accuracy, and cannot test it as I will be away from my computer for the next few months

Everybody has to do debugging when their scripts don't work, as it is so easy to make one little mistake, causing the whole code to make no sence. The first step in debugging is to switch the in-game debugging options on. To do this, open up Civilization4.ini (make sure to back it up), and change the following entries to the following values:

HidePythonExceptions = 0
ShowPythonDebugMsgs = 1
LoggingEnabled = 1
MessageLog = 1

This should enable in-game python popups, and cause all errors and messages to be printed in your My Documents/My Games/Civilization 4/Logs directory. The three important logs to look for when debugging are PythonDbg, PythonErr and PythonErr2.

Common error messages and their causes:

In the logs usually the errors will have a traceback on them. This traceback lists all the files that are effected by the error, and the lines the error occur on. The lase line is usually the one you are interested in, as this is the one where the error occurs. Sometimes you'll have to go back a few steps as the error won't be on the last line, but usually it is!

Syntax errors:

These are mainly typing errors - missed ":"s, incorrect whitespace, or bad bracketing. The game usually picks up on these when you load it up. The error logs should show you exactly where the error is in the syntax with a little ^ symbol underneath it.

Argument errors:

If you try to use a function designed for a pointer on an integer value, you will get a type error. Go to the line specified and check that the function you're are using is valid. Usually it will say what type of function it is expecting, and what you've given it. NOTE: Some classes need () after them in functions, some don't. If you are getting a Type Error this may be the cause.

Type Errors:

These either occur when you are passing a function too many, or too few arguments. You have to make sure the amount of arguments you are passing a function matches the amount of arguments it takes, or the function will throw up an error when you try to run it.

Name Errors:

This means that you've tried to use something that the game doesn't know what is. For example, if you were to type a = b without first saying what b was, there is no way the computer could set a vale to a. Remember a=b does not do the same as b=a. Also, make sure you haven't any "==" where you want "=", or vice versa.

Sometimes you'll get "Argument referenced before assignment". I don't know if this goes alongside the Name Error, if it needs it's own category, but basically it's like a name error, but you define "a" later on in your code.

"List index out of range"

This happens when you try and reference a index in a list, turple, or dictionary which doesn't exist. This one can be tough, and you might have to use methods described in the next part of this section to see exactly what is going wrong.

That's all the error messages I can think of right now. I'm pretty sure there are one or two I've forgotten - if you find any please feel free to ask and I'll include them here. As I said at the start I'm not 100% certain on the accuracy of these, but they should be ok.

Code not behaving

So you've written your code. It all functions with no errors... but it doesn't actually produce the results you expect it to in the game. There can be several causes of this:

Firstly, if you think a piece of code should be having an effect in the game, make sure that it is by checking the API to see if it is a VOID function. It's quite a common mistake when first starting out to use other functions thinking that they will do something, when in fact they just retrieve information.

Secondly, if everything seems right you need to start putting debug messages in. This section brings you back right to the very start of python coding, when you were telling your computer to print "hello world". What you will need to do is to get the computer to print a message telling you what a certain value is at a certain point in the code, and compare it to what you think it should be. For example, the following code will print "a=b" in the debugging log if a=b, and print the values of a and b otherwise:

if a == b:
	print "a=b"
else:
	print a
	print b

This can be quite hard to decypher in the debug log (often you will have many many numbers), so you can do more advanced messages

if a == b:
	print "a = b, a and b are %d, and %d"%(a,b))
else:
	print("a = %d"%(a))
	print("b = %d"%(b))

Where %d will reference the number outside the "" after the %. If you want to place a string there you have to use %s. This can give you detailed information as to exactly what is going on inside your code, and hopefully from that you should be able to see what is going wrong with it.

Misc

This page contains other stuff about Civ 4 python that isn't really covered in the other posts, and is not obvious at first glance. Some of it is fan-made - creating shortcuts to do some quite complicated things.

PyHelpers.py:

Most python files import this file. Basically it adds a few shortcuts to some more complicated functions. If you can't find exactly what you want in the API there is a chance that this file might be able to help you - though it only really creates shortcuts, and still follows the API.

CvGameUtils.py:

This file is used when deciding certain things, like what can/cannot be built. It can be used to interrupt some basic game functions. Normally each function returns "False", but if you were to make it return "True" under some certain conditions on those conditions the unit, for example, could be removed from the list of units available. You could use this to make having the slavery civic make a slave unit be buildable, for example. I'd check it out yourself to see exactly how much power you have in there.

Scriptdata:

If you want to add additional information about certain parts of the game world you need scriptdata. For example, if you want a unit to use up "ammo" everytime it fought, and then have to return to a city with an armoury to resupply, or maybe you want plots to become muddy if too many units travel on them without roads, you need scriptdata.

If you want to play around with scriptdata, I recommend getting Stone-D's excellent SD-Toolkit. This toolkit allows you to attach pieces of data to parts of the game. Each part must be identified with a unique mod-name (for compatibility).

NOTE: There is an improved version of the toolkit posted on this post on CFC by Teg_Navanis. This version is faster than the default version, and fixes several minor issues the default had.

Action Buttons:

talchas has released an Action Button's Utility Mod which is a template for adding buttons to the GUI to do custom functions when pressed. The AI will have no clue about it.

Square Selection:

Once again by talchas this mod is a template. With suitable modification this could be used to make artilliary act exactly as Civ 3 artilliary, with no bodges. Once again, AI shortcomings come in.

.ini file modifications:

While I have not actually tried it myself, there is a mod by Dr Elmer Jiggle which allows you to add variables to a mod's .ini file. Get it here

Contents