Bang and Blame

The Joy of C++

The joy of C++, an catalog of some ways C++ programs can and do fail in practice. 

Error Handling via Exceptions

C++ supports an exception mechanism - errors can be reported by throwing an exception, and handled by code catching the exception higher up the call stack. Given that C++ is, well, C++, you can throw values that are not exceptions but of any desired type.

Consequently, it is hard to know what exceptions should be handled - there is no common base type. You can try "catch (...)" but then its an interesting challenge to portably determine what the unknown exception is.

The cost of missing to catch an exception is program termination. So, it would be nice if the compiler could assist in ensuring that exceptions that can be thrown are in fact caught.

The compiler will not assist. Functions can have exception specifications, stating either that "nothing is thrown" or "A or B is thrown". These are not statically enforced, but dynamically checked: the action on failing the check is, essentially, to terminate the program. 

Such checks may interfere with the ability of compilers to optimise, and certainly complicate the process.

 

Not all exceptions are Exceptions

The exception handling mechanism of C++ is inadequate for another reason - not all errors generate exceptions. There is no Java-style "nil pointer" deference exception in C++. In broad strokes, your program will crash on dereferencing a null pointer. Such an error is a "system thing", as is integer division by zero. The behaviour is "undefined" - which means in practice you cannot reasonably handle it, portably. 

 

Disabling Error Handling via Exceptions

The C++ language is often subset - in the name of runtime efficiency - by disabling exception handling (typically together with run time type information (RTTI). The consequence is that code must then either return errors via return values, or locally decide that an error is fatal and invoke "abort()" or "exit()". 

Error codes may all to easily be ignored. The other case, where code may locally call "abort()" on error requires that all callers be aware of how all the code they call, directly or indirectly, will handle errors.

It is not best practice for one dynamically linked library in a large application to terminate the whole, with no chance to intervene in the error handling. Yet, without careful API design in all the intervening layers, it may not be possible to report the error all the way up to the top layer.

Stack Overflow

The call stack can overflow in any language with recursion where stack allocation is used for activation records and tail call optimisation is not mandatory. So, this not a C++ specific problem. Yet, C++ does not provide any in language mechanism to know if you *will* overflow the stack, or to handle if you *do* overflow the stack.

 

Two approaches seem used, based on how the program is linked. You can place the stack at the top of memory, and grow the stack towards the top: exceeding the available space will then trigger a pagefault (or memory protection error) and terminate the program. Or you can have a guard page at the top of the stack area, to trigger either a memory protection fault or to read from and detect overwrites after the fact.

 

The stack may grow towards the heap area - leading to potential memory corruption on overflow. This may lead to hard to diagnose bugs that do not occur in "normal" usage.

 

The desire for efficiency in C++ prevents a runtime system being widely used that would bound the call stack depth - as interpreters for e.g. Python do. Similarly, a calling convention with call stack checking is also rejected. In such a scheme, calls to functions would check the remaining size of the stack area, and either pass the size remaining, or compute it on the fly.

 

C++ is "portable". It does not mean that your program will work the same when ported - even on non resource constrained systems, the stack size is variable, as is the allocation size of each stack frame. Your program may fail with stack overflow on some, but not all, platforms: and at some, but not all, optimisation levels.  

 

If the stack is large enough, stack overflow may be a seldom seen failure only occurring from unbounded recursion, typically from a logic error. It can also be provoked, if naive recursive code must process user supplied inputs for e.g. parsing languages with nesting constructs. 

Memory Allocation Failure

C++ heap allocation functions can raise exceptions. Not a problem specific to C++, especially, but hard to recover from. What to do when you have no memory left? What to free? What code is safe to run, and will not attempt itself to allocate more? In general, when the failure to allocate occurs, it is easiest to terminate the program.

There is not a single approach to heap allocation. You may find allocations in C++ style, C style, and in the normal heap, or in arenas of buffers with placement new, or in system heaps. The diversity makes uniform treatment of exhaustion difficult.

The wrinkle with C++ is that allocations are often manual, with sizes in integer types: integer types subject to, for example, overflow: or where a negative signed number is interpreted as a very large unsigned number. Thus, it can be seen that some memory allocation failures are due to attempts to allocate extremely large amounts of memory due to an incorrect size resulting from confusion of signed and unsigned types. 

 

If taking sizes from external inputs or data, it may be worth checking they are plausible before allocating.

Memory Corruptions

C++ is not memory safe. You can write past the end of an array, or the start; that can corrupt the stack or heap, leading to unpredictable behaviour depending on what data or code got corrupted.

The fun aspect is the distance between corruption and detection, and the potential for silently corrupting data.

In Silence

Assertions are good! They should be used. Just, not with the anti pattern of asserting that something must hold, and then, in a release build, merely proceeding as if all was well down a code path that would have been aborted due to the assertion failure in a debug build.

ABI (Application Binary Interface) mismatch

C++ is compiled by multiple compilers. To interoperate, the compilers must agree on, amongst other issues, the size of data types, and the layout of data values in memory and when passing arguments to, and values from, functions, the placement of those in registers or stack locations. 

When a mismatch is not detected at e.g. link time, it may lead to hard to diagnose runtime errors. The outcome of ABI mismatch may be memory corruption, or the corruption of the arguments passed to a function.

Thread Lifespan vs Scoped Lifespans

Managing lifespans of scoped data in C++ is eased by RAII - destructors being run when execution leaves a scope. The issues with threads are not so simple: what to do with a thread that has lived "too long"? You cannot portably, simply, or safely kill it. You cannot let it run forever. You can ask it nicely to exit, if there is a mechanism to do so - but then - how long shall you wait?

If you destroy an instance of the C++ thread class without having called "join()", standard terminate is invoked - your program abruptly exits. 

 

Various thread usage patterns do this: the problem being, that threading and concurrency are hard to test, and timing can affect the order of operations and therefore problems that occur only under load.

Live Lock, Dead Lock, all the usual suspects

The standard mutex type in C++ is not a recursive mutex: a thread can deadlock itself by taking the same mutex twice. As usual, the advice is to not make that kind of mistake. 

DLLs, Libraries, Plugins, and Lifespans

C++ semantics for construction and destruction of globals interact in complex ways with dynamic loading and unloading of shared libraries, and with reloading. It is unsafe, in the extreme, to pass references and pointers to data in one DLL to another, without care for the lifespans of the DLLs contra data. 

And in all the normal ways programs can fail

C++ programs can fail for reasons not specific to C++ - logic errors, typically. There is a sufficient number of ways to fail, that we may wish for fewer added by C++. 

 

 

Last Man Standing

Thanks to OpenXCOM (https://openxcom.org/) I have been replaying the XCOM 1 (UFO - Enemy Unknown) and XCOM 2 (Terror from the Deep) games from 1994 + 1995. Thus, with the aid of 25 years further gameplay experience, an improved game engine, bug fixes, mods and only modest amounts of save scumming I have completed XCOM 1.

It is done on Windows - the first time in 20+years the home PC has not been a Mac or a PC running Linux. The cost effectiveness of buying a Mac is simply no longer there, and who wants to administer a Linux system any more or try to diagnose why power management is not all it could be? So far, Windows has not been majorly annoying. It is certainly convenient that a Steam install of the XCOM games is easily found by OpenXCOM.

The reviews of TFTD have rarely been positive in comparison to its predecessor. It has been suggested that the preference for TFTD depends on playing it first, before XCOM. I may agree; I played TFTD before I ever got a copy of XCOM, and found XCOM somewhat lacking in comparison to TFTD for various reasons.

Firstly, TFTD is a reskin of XCOM except with a decent budget for art - both graphics and sound. It has a larger range of animations, more varied level terrain, richer artwork in research reports, and crucially, an oppressive, brooding sound track that provides eery punctuation to the murky depths of underwater combat - combining cautious exploration and reckless attack. The graphics are basic by todays standards, but not spartan or cartoony, as could be said for some of the XCOM artwork.

Both games had bugs as shipped, which are now forgotten and fixed in the remade game engines (and patch releases made in the 90s, too). Yet, the interface of TFTD added one important aspect; opening a door without immediately entering the room or area behind it. This is quite important when the next room can be empty or full or heavily armed aliens...

The difficulty of TFTD is a response to the infamous bug of XCOM where all the save games reverted to "easy"; thus, the difficulty levels actually work, and are harder, too. The increased difficulty of TFTD leads to consequences, and tension, in the game that are not so evident in XCOM. There is not an agreed on set of tactics applicable in nearly all scenarios for TFTD, as there is for XCOM: and those tactics that do exist are not as certain as for XCOM. It is not unlikely to suffer heavy casualties on even small missions. The game does not become a walk in the park in the same way that XCOM does.

TFTD seems to be able to break the routine of XCOM missions, which could be described as first disembarking, then scouting and locating the UFO, before finally storming the UFO in an assault to end the mission. This seems to be done by various means. One is by having the landing site close to the downed alien craft, leading to a choice: to start an assault, and risk being flanked, or to attempt to contain the aliens within the sub while clearing the area? 

Clearing the area is made considerably more difficult by more complex and 3d terrain - its no longer an option to have every soldier in the squad, even without line of sight, shoot at each spotted alien in a massive firestorm to mow them down one by one. Instead, line of fire is often blocked, and the encounters are often down to one or two soldiers versus one or two aliens.

Larger levels lead to an increased risk of being outflanked: tougher aliens also lead to more re-awakenings of stunned aliens that were not killed outright. 

An emphasis on hand to hand combat requires the fragile soldiers to get close to the monstrous creatures to battle them; no more merely hanging back and relying on volleys of automatic plasma fire to prevent proximity. The scariest alien of XCOM (the Chrysallid) is back, except this time it flies (swims!) and no flying suit will save you from it now. The most powerful weapons are not available for all missions (above vs below water) so cannot be leant upon as crutch.

In any case, the games are both now entirely moddable, so you can turn off aspects of the game that irritate (PSI! money worries!) and add fun new behaviours (spherical explosions, better base layouts) and enjoy the bits of the game you want.

Now, I have to salvage the situation of 70% casualties on the first encounter with Lobster men! Back to saving the world

Re-Arranged

Thanks the the forums for the raspberry pi, this keyboard + track pad combination just got a lot less irritating. Disable tap to click.  For Logitech K400 keyboards, hold down FN, left click on trackpad - disables the tap to click. 

HDMI auto detection seems to choose the highest resolution possible, regardless of how bad that actually looks on the TV

Override in config.txt

hdmi_group=1

hdmi_mode=19

The Rest Will Follow

The LLVM + Clang compiler toolchain 3.6 release can be downloaded as prebuilt executables for Windows. So far, so good, so what? The interesting and practical part of this is the source and ABI compatability with Microsoft Visual C++ and the integration with Visual Studio. This means that choosing to use the Clang front end and LLVM backend is as simple as choosing the relevant toolchain in the VS properties (ideally, at the configure step of using CMake to generate the VS project!). This painless process means it is trivial to try out Clang without needing to be steeped in Linux, or owner of the expensive dongle for OSX aka some sort of Mac computer hardware.

Reality is, of course, that this does not yet provide an actual satisfactory outcome. Clang + LLVM on Windows for the VS ABI is still a work in process, with respect to not handling all of the extensions to C++ present in the Windows SDK and MFC headers, and the lack as yet of any support for C++ exceptions.

None the less, the compiler functions as a useful and valuable linter of C++ code previously restricted to building with MSVC due to heavy usage of MFC and Windows headers. 

I Know Jack

An interesting and very readable ramble through various aspects of ARM assembly programming.

1. Introducing ARM ASM on Raspberry Pi
2. Registers
3. Load and Store
4. Debugging ASM with GDB
5. Branching
6. Control Structures
7. Shifted Operand
8. Arrays and Structures, Indexing Modes
9. Functions
10. The Stack
11. Predication
12. Control Flow, Condition Flags, 64 bit Integer operations
13. Floating Point
14. VFPv2 Matrix Multiply
15. Integer Division
16. Control Flow via Switch
17. Function Arguments and Return Values
18. More Stack Handling
19. Interacting with the OS
20. Indirect Calls
21. Subword Data
22. Thumb Instruction Set
23. Nested Functions
24. Nested Functions + Trampolines

Silent Scream

Stopping KDE applications printing useless and distracting debug text to stdout / stderr and polluting the terminal window used to invoke them.  Charmingly described as stopping KDE apps from vomiting all over your Xterm. 

In short; run kdebugdialog and click the deselect all all button.

A Moment Of Luck In The Sea Of Misfortune

The latest update to Microsoft Visual Studio 2012 gives the ability to build C++11 programs for versions of Windows from XP upwards. For that build configuration, the static analysis tools are disabled, as are some other areas of less used functionality (HLSL compilation, remote debugging, direct x debugging). Note the new v110_xp mode is the only supported one for targeting XP.

This new update requires updates to tools around MSVC e.g. CMake 2.8.11 is required to use the new mode. 

Updating MSVC also brings new advice about what compiler options are sensible, and what options are newly harmful. /Za is bad in MSVC 2010+, and is discussed in the boost warning guidelines.

Final Chapter: Thus Ends...

The reason why gdb would not work on Ubuntu: a deliberate policy by the OS developers. Cheers, guys. At least the stack overflow(s) make finding why easier.

There is a lot of good looking "web stuff" around. 

Sublime Text 2 is good in part because of packages (installation, usage). Also as it works in Linux, Windows, and OSX.

Nostalgic returns to operating systems of the past require the opening of disk images as floppy drives are in the past (as indeed, the disks...). A RISCOS disk module to make disk images appear as mountable disks exists, and is useful in ARCEM.

Once again, I content myself to computing on OSX and Windows, leaving Linux again except on the Raspberry PI. The usability issues are never addressed adequately. Its an open question if the pain points of the past are addressed before it becomes irrelevant e.g. CD/DVD burning may finally work well, but do I care any longer? Some old and fun rants on Linux usability (or the lack thereof).

The Nature of Code is a fascinating book. 

IDE testing is not so often blogged about.

Word games are popular, but need dictionaries! Here is a game with swedish and english dictionaries in Sqllite DB files. (May be a good game, don't know!)

Not Ready Yet

Fired up my copy of iA Writer, got pandoc to make some HTML from markdown, and wrote up my toy interpreter with its byte code to OpenCL compilation. Pandoc is nifty, and can generate pretty code snippets in HTML with syntax highlighting and line numbering.

The More I See

It is possible to use printf inside an OpenCL CPU kernel on OSX; however, remember to cast the string literal operand to (char const*) pointer to get it to compile.

Further, when running an OpenCL using program it is possible to force error logging to stdout (e.g. for .cl compile errors) via 

    CL_LOG_ERRORS=stdout program Args 

Disassemble

There is an offline compiler for OpenCL included in OSX 10.7, at /System/Library/Frameworks/OpenCL.framework/Versions/A/Libraries/openclc 

This is pretty much a standard clang executable, and accepts common clang options. Sample arguments of interest, 

    -O3  -x cl -triple i386-applecl-darwin -S < loop.cl

Shows the assembly code for an OpenCL kernel.

    -x cl -triple i386-applecl-darwin -emit-llvm < loop.cl

Shows the LLVM textual-IR for a kernel.

The official documentation and example of this suggest the use of generating LLVM bit code offline, for 32 and 64 bit CPU and 32 bit GPU. More useful advice included in the OpenCL Mac programming guide.

Enough Space

The point of a RaspberryPI is, largely, to program it. Getting your code from the PC onto the PI, building it, and getting it back from the PI to PC can be done easily with hg (aka mercurial), the distributed version control system. This makes headless usage of a PI over a network via ssh a bit easier. Mercurial is readily available to debian based linux on the PI via    

    sudo apt-get install mercurial

There are two aspects of this: getting code onto the PI, and from the PI, pushing the code / build artefacts back to the original machine. It seems sensible not to trust a cheap SD card overmuch.
Mercurial includes a builtin webserver that can permit remote access to your code, both read access and write access.
Assuming you have already an hg repository created, you can serve this as follows (from the hg repository directory) on your development PC

    hg serve

You can then view the repository in the web browser at http://localhost:8000

This is also now visible to the PI (and any other machines on the same network, not a secure practice!). 
The initial clone can be done as follows

    hg clone http://192.168.1.71:8000 destdir

Subsequently, changes made on the PC can be fetched via hg as well, from within the hg repo dir on the PI

    hg pull

Since the PI is a full Linux machine, its quite practical to make changes locally on the PI (say, from an ssh session or remote X11 setup). By default, however hg will not let a push occur to a repository served via hg serve

This is possible, however. On the PC (in the .hgrc or Mercurial.ini file, depending on OS) add the following section.

   [web]
   allow_push = *
   push_ssl = false

Changes made on the PI can then be pushed via hg, from within the hg repo dir on the PI

    hg push

Counting

The RaspberryPI is a small, credit card sized, ARM based PC that can run Linux. Love2D is a game framework for 2D games in Lua, with physics via box2d and all round quite nice for writing small games in across Windows, OSX, and Linux. 

I have built the current Love2D on the RaspberryPI using the current suggested release of Linux for the pi (based on Debian Squeeze). This was just following the instructions on building Love2D from source at the Love2D wiki. There is an outdated build of Love2D in the debian repo but this is 0.5.0 vs the current 0.8.0 release.

Current status: the demos seem to run, just very, very, slowly. This is probably attributable to softfp being used throughout the whole OS build, and non-optimized X11 drivers on the PI. With luck, all that is needed to make Love2D good for use is the system being built with hardfp as is done on the raspbian project.

Having a blast

Random notes on the installation of Linux - if using a rotated screen, and can get an Xterm from the live-CD/installer, then you can use 
  
  xrandr --output VGA1 --rotate left

to not hurt my head + neck - except when it boots or in the BIOS.

If it complains about a missing bits/predefs.h in building anything, go back and install libc6-dev

   sudo apt-get install gcc-multilib libc6-i386 lib6-dev-i386

Of all the fractured X11 GUI nonsense in Linux atm, I really don´t know which one I got in Linux Mint - other than its some variant of GNOME 3. I guess that I don´t care means it works better than Ubuntu unity...

The Game Has Changed

LLVM is described as "a collection of modular and reusable compiler and toolchain technologies". While true, this description is insufficient:


It is modular, insofar as one can pick and choose from the collection of technologies present within LLVM at a given moment in time. Either from a stable release, or a snapshot of the SVN trunk. Neither approach is satisfactory.


The LLVM project encompasses a set of libraries to manipulate program representations (e.g. LLVM IR), tools to operate on programs in IR form, compiler backends to translate IR to target specific machine code, and with the Clang subproject, a compiler front end to translate programs written in C, C++, Objective-C, and OpenCL into IR. Within this system, some new behaviours may be added through addition of submodules to add, for example, a new pass over IR.


The pace of change in the LLVM SVN trunk is rapid. This includes changes in aspects from the naming conventions for APIs to fundamental alterations in the IR type system shared throughout LLVM, to deprecation of entire compiler back-ends, APIs, and the replacement of those. The laudable goal is to rapidly advance the quality of LLVM, and not to be hampered by approaches found insufficient in some aspect, nor by code lacking an active maintainer. Like any active development project, regressions will and do occur, and the trunk stability fluctuates in any metric from correctness, performance, quality of generated code, or buildability: regressions are intended to be found by testing on build bots, and minimised by a review process on commits either before or after commit.


Periodically, at about 6 month intervals, the code is stabilised and a new "stable" release is made after a period of testing and regression fixes. This code, once tagged as "RELEASE N", is then not updated. Not for trivial bug fixes, not for serious bug fixes, nor the addition of entirely new functionality.


Thus, the choice for a user of LLVM (a user being either a developer using the LLVM compiler suite simply as a compiler, or a developer building upon LLVM to create compilers or other language tools) is to live with a stable release, that rapidly falls behind the current version, or a current version that constantly changes and makes no quality guarantees.


Finding a bug in a stable LLVM release is hopefully rare. It is not impossible, however. The problem is then, when a bug is found, a fix for it is not applied to the stable release. Instead, it is applied to trunk (if present there as well) and becomes available in the next release at some point in the future. If you want that fix on the current (or a previous stable release) you need to cherry pick the fix, and backport it. The difficulty is probably proportional both to the size of the fix, and the duration of the interval between the release date of the release(s) affected and the current date.


So, if a bug is found, a fix will not become available in a simple update to a stable release. There are no patches. There is no LLVM 3.0.1 to follow 3.0. Nor will fixes be backported to previous or older releases, which themselves may not be very old e.g. in a codebase less than 1 year diverged from the tip of the SVN trunk.


Further, while LLVM and related projects are modular, and build on one another, they are tightly coupled. The releases of Clang are tied to the equivalent releases of LLVM. (The trunk build of Clang and LLVM will typically fail to compile if permitted to get of sync). This will assuredly also apply to other LLVM subprojects such as LLDB. You cannot mix and match between releases. You cannot have a new Clang release, with new language features e.g. C++11, and an older LLVM backend (e.g. Alpha just got removed, the C backend is increasingly unmaintained). This would require at least a commitment to a stable API - which LLVM will not provide. (A stable ABI between C++ libraries is an inherent challenge in any case).


To add a new module e.g. a backend, you choose a version of LLVM at a moment in time and implement code generation in the context of that LLVM. If you choose trunk, you will have a constant stream of mostly minor changes to deal with, with the exceptions of subsystem APIs being iteratively improved or changed, minor build system issues such as makefiles being broken, fixed, and changed, and entire modules being removed. If you choose a stable release, you forgo ongoing bug fixes and improvement: and face a massively disruptive change in porting to the next stable release, unless you stick with your old version. The LLVM release notes indicate both major changes and removed features, and the internal API changes: even incomplete listings of these are lengthy, and adaptation of existing code to a new release can be anywhere from trivial to exasperating.


The LLVM infrastructure permits many ways of adding new behaviour to LLVM - plugging in a new pass, adding a new backend: but it is impractical to add all desirable changes in a clean manner by plugging in a new module, as the number of hooks via which change can effected in this manner is limited. At times, changes need to be made within LLVM components (either as dirty hacks, or as the sole means to achieve some legit technical goal). Preserving these changed components is hard: the code in which changes are made will evolve over successive releases, and the modified component cannot be dropped in to a different LLVM release. 


To sum up: it would be desirable if LLVM provided a) backports of bugfixes to stable branches, instead of periodic release snapshots of code and b) stable APIs between components, to less tightly couple the components of LLVM to one specific LLVM release.


The modularity of LLVM + related projects is impressive when compared to the monolith of GCC: LLVM posesses a decent architecture, implementation language (tasteful C++ subset), and a t least one portable and relatively comprehensible build system (via CMake). It would be nice if point releases were made to patch known bugs, and API design was undertaken to loosen inter module dependencies and ease ports of subsystems between LLVM release versions. 


None of the above are insurmountable obstacles to the use of LLVM - but I blench at the prospect of porting changes, features, and fixes forward repeatedly, again - or of living with constant API and design churn.

Hanger 18

Back to early 90's - XCOM aka UFO returns (via an open-source remake). Happily, nearly trivial to build on OSX.


Grab the source. Github has a nice OSX client, and the webpages have a wee button to spawn the OSX client and make the git clone process painless.


Get a packaging system for install Unix packages on OSX - the new hotness is Homebrew, in addition to fink or macports. Homebrew has the relevant dependencies, so it works (tm).


brew install yaml-cpp
brew install sdl
brew install sdl_gfx
brew install sdl_mixer


set the pkgconfig tools environment variable to find the installed stuff.


export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/lib/pkgconfig


In the src directory of OpenXcom


make


And tis done. Copy the games data files to the relevant location, and all is well.

Terminal Illusions

Flocking makes for a nice screen-saver - see here for a nice JS demo, and this pseudocode for a nice, dirt simple explanation. Managing so many objects interacting in space, even 2D, calls for some spatial data structures. The approach of buckets in 1D, and sorting + binary searching in the other dimension seems to work well for my flocking screen saver. 64 bit in Lion (hmm. and Snow Leopard?) break 32 bit screen saver modules (bah!) and also Saver Lab. Can capture screenshots with another OSX command line tool, screencapture.

Simple Flocking ScreenSaver

GDB in Visual Studio - they are working on an Android NDK version,
which could be a great improvement on current tools, if it worked.


The C++ language has plenty of odd corners; but so do the standard libraries, and the cross platform variations. The std::string class can surprise on some platforms with its Copy On Write semantics and the sharing of underlying memory buffers.

Every new version of Windows breeds a new bit of software to handle disk images, see WinCDEmu for Windows 7.


Getting a visa to travel to the states can be done online. The EU should reciprocate with US visitors until this nonsense gets stopped. 


Consider Lockharts lament; replace "maths" with "programming", and ponder on education now. And recall how maths was taught, too. Ick.


The HunSpell spell checker has many dictionaries. The executable can be controlled via pipes from Java. There is a Swedish dictionary from "The Big Swedish Wordlist".

Not Ready Yet

ConstCPP - C++ with optional mutable state aka const by default.

This blog posting ui is nearly unusable on iPhone - been a while since I found a site with this sort of issue.

Sleepless Nights

To quote a colleague

I want gdb to die in a fire. If thats possible.

A sentiment I can only agree with, to be honest. However, it remains in its spread of variably buggy obsolete forms embedded within the development tool chains of too much of the world to avoid. Some notes on living in the land of the clone of the Unix tools from the 1980s. A link to a GDB reference card. Programmable systems (debuggers like GDB, editors like emacs, typesetting systems like LaTeX) have been universally dire in user experience, in _my_ experience. The intended bonus of programmability becomes a curse of necessity; the burden is placed on the user to make the tool usable in the first place. In essence, I want to write programs to solve _my_ problems; not to solve problems in the tools I use; and _not_ to write programs in the debugger while debugging the interesting program or to debug program errors in a document processor.

A set of optparse documentation for common tasks since, as it notes,

we start new project so seldom, it is difficult to remember all those recipes that we come up with, every time we add a new command line option. This article is an attempt to bring together such recipes.

Python can query its environment for the functions in the current module (which is done slightly differently than for _other_ modules). This is useful for auto-testing, saving the need to write out a function list explicitly. Can embed these gist things into a blog post, too.



After the GDB rant, I confess I don't program gdb much and have not (yet) needed a .gdbinit file to customise GDB in a semi-permanent manner. However, I have some user defined commands to not forget in case I need to stay in GDB a little longer...



JGrowl implements growl style notifications in a web page. Cool, if not immediately useful. Also, drop shadows via CSS.

Can fork multiple threads (well, processes) in bash and then join (wait) for the completion of them. Good for parallelism of shell script controlled processes.

cmd_A &
cmd_B &
wait

Left Unfinished

Can build a simple and useful unit test system in python exploiting first class functions, dynamic checks on the names of functions, and a naming convention.


See codepad.org for coloured src and a sample run, or github for a "gist" with highlighted source.

### Begin unit test code


def test_function1():
    # Do some test code
    # return True on Pass, False on Fail


def test_function2_EXPECTED_FAIL():
    # Do some test (more) code
    # return True on Pass, False on Fail


def test_functionN():
    # Do some test (more) code
    # return True on Pass, False on Fail


# Build a list of test functions to run
# as test cases
tests = [ test_function1
        , test_function2_EXPECTED_FAIL
        # as many as you like...
        , test_functionN
        ]


def main():
    failed  = []
    # Invoke each test case in turn
    for t in tests:
        if not t():
            # Can check names of test functions 
            # to detect expected fails
            if t.__name__.endswith("_EXPECTED_FAIL"):
                continue
            # Record fails for later report
            failed.append(t.__name__)
            # Can stop on 1st fail if desired
            # return 1
    # Print the list of tests that failed, if any
    for f in failed:
        print "FAIL:",f
        
# Invoke the test runner
if __name__ == "__main__":
    main()

### End unit test code

Tile based game development tutorial.

A series of blog articles on Maze generation algorithms.

A C pre processor (cpp) in python. A programmers text editor Kod for OSX based on the google chrome tab UI.

Lore of the Arcane

Compilation of Offload C++ code for the Cell BE processor and the Linux OS entirely within a Linux environment is complicated by the fact that the Offload C++ compiler ("offloadcpp") is a Windows executable. This compiler compiles programs to run on the Cell processor by first compiling Offload C++ code into Cell specific C code; the C is then compiled for the Cell processor using a Cell BE GCC tool chain (GCC and associated utilities) on Windows, via Cygwin. The result is that, on Windows, the compiler can generate a Cell Linux executable with embedded SPU sub-programs automatically.

The below process (described by Paul Keir, thanks!) outlines how the Offload C++ compiler can be used to compile for the Cell from within Linux using Wine. The first stage is to compile the Offload C++ code (your original program) into C code targeting the Cell. The second stage is to compile that generated C code on the Cell using the Cell SDK on the PS3 running Linux.

Note that depending on how your Linux PS3 is setup, it is possible to create a script (e.g. via scp and ssh) to push the generated code onto the PS3, and to subsequently invoke the PS3 compilation process on the PS3 automatically.                                 
                                             
--------------  Stage 1: Linux/Windows ------------

1.  Install Wine on your Linux system.

2.  Copy the Codeplay Offload C++ installation directory from Windows to somewhere convenient on your Linux system.
   
3.  On Linux, set the SIEVESDK environment variable to the offload-cell-linux-sdk directory contained within the installation directory.

4.  Offload may install a Cygwin-compatible Cell Toolchain. By default it will be installed in C:\cygwin\opt\cell .

Place the entire cell directory tree within the /opt/ directory of your Linux system. The Cygwin-compatible PS3 toolchain is also available here: http://sourceforge.net/projects/cellwindowssdk/

The toolchain.7z file there also contains the /opt/cell/... directory tree.

5.  The Offload compiler is now ready to run, though only with the -nomake switch. If you have placed the Offload installation directory at $HOME/apps/codeplay, offloadcpp may be used to compile hello.cpp using the following command:

$ wine $HOME/apps/codeplay/offloadcpp.exe -I$SIEVESDK/include -I/opt/cell/toolchain/include/c++/4.1.1 -I/opt/cell/sysroot/usr/include -I/opt/cell/toolchain/lib/gcc/powerpc-linux/4.1.1/include/c++/powerpc-linux -I/opt/cell/toolchain/lib/gcc/powerpc-linux/4.1.1/include -nomake hello.cpp

An alias may be useful, say "oll":

$ alias oll='wine $HOME/apps/codeplay/offloadcpp.exe -I$SIEVESDK/include -I/opt/cell/toolchain/include/c++/4.1.1 -I/opt/cell/sysroot/usr/include -I/opt/cell/toolchain/lib/gcc/powerpc-linux/4.1.1/include/c++/powerpc-linux -I/opt/cell/toolchain/lib/gcc/powerpc-linux/4.1.1/include'

$ oll -nomake hello.cpp

6.  The result will be a directory called "outputc". This directory must now be transferred to the PS3, before issuing the make command, as described in Step 3. below.

--------------  Stage 2: PlayStation 3 ------------

1.  Copy the offload-cell-linux-sdk directory to the PS3, and set the SIEVESDK environment variable to that location.

2.  Mars should already be installed (mars-1.1.4-1.ppc.rpm), but the development libraries are now also needed, so:

$ wget ftp://ftp.infradead.org/pub/Sony-PS3/mars/1.1.4/mars-devel-1.1.4-1.ppc.rpm
$ rpm -ivh mars-devel-1.1.4-1.ppc.rpm

3.  An "outputc" directory, obtained as in step 5. above, contains a makefile. To complete the compilation of hello.cpp:
    
$ cd outputc
$ make
$ cd ..
$ ./hello

What comes around

Lichess is a cool online chess site. Although, it is not, as it claims, open source: it places non-commercial use restrictions on the code, for example, failing the open source definition.

Its easy to write simple web applications quickly with Webpy. Can parse C99 with open source parser (pycparser) implemented in Python, and display the generated syntax tree in a web browser via JSTree. Hence a very semi-working OpenCL lint tool (just needs, hmm, a proper preprocessor, symbol table, type-checker to do this right: not going to do that outside the day job, obviously!).

The graphics support in browsers is getting better: Graphviz display in the browser via canviz, and some ease of use APIs over the HTML5 canvas (EaselJS) element.

Maze generation. Nifty CS assignments I wish I had on my course. Key events in JavaScript. If this script works, and the web-service it uses is any good, finding plane tickets should be easier.

With the somewhat more extreme climate around, weather seems more interesting. See also wview.

For journalism, and print media that 'gets' the internet, the Economist is pretty good; a print subscriber gets the same content online, and in an iPhone App, and in audio. It's rather like distilled BBC Radio 4.

Cloning Technology

Concurrent Programming with Revisions and Isolation Types from Microsoft Research (here) is an approach to concurrent programming looking extremely similar to Sieve C++ (described here, here and  here) developed at Codeplay.


How to setup virtual box shared folders to access host data from the Linux OS inside the VM after installing the guest addons; it works, but is very much slower than a real machine.


Intel have released an alpha OpenCL implementation (for X86).
There is also an interesting performance guide, SIGGRAPH slides, noting that transformations to code beneficial on GPU are not optimisations on X86 (e.g. exploitation of hierarchical memory spaces via __local etc).

Crashing Around You

One who seeks knowledge learns something new every day.
One who seeks the Tao unlearns something new every day.
Less and less remains until you arrive at non-action.
When you arrive at non-action,
nothing will be left undone.

The above is a commit message: I wish it had been informative, not pretentious.

Its time for operating systems to prohibit the space character in file and directory names. Too much breakage with respect to quoting even today, and this is a classic known issue in programming since before I was born.

Ubuntu 10.10 installs pretty straightforwardly in VirtualBox, but you need to install the "Guest Addons" from Virtual Box to teach it that there is a monitor capable of more than 800x600 resolution. Sony Vaio laptops need the virtualisation of the hardware enabled in the BIOS to let VirtualBox run the VM it creates: thats the first BIOS I have seen in, four? years or more. F2 gets the BIOS up at boot, and the enable virtualisation option is there. Note that this is after the BIOS update, that obsoletes all the dangerous hacks on the net that talk of unofficial BIOS updates / patches, that are now sitting around on the web to break careless readers.

Ain't it Fun

Teaching modern C++, as opposed to warmed up C in Modern C++.


Scripting user interfaces with images in Sikuli. Interesting, if inherently brittle.


Isometric in browser game development in JS and HTML5 canvas,


Pyparsing - some or all of the power of Parsec without the brain hurt of Haskell. MARS : a MIPS assembly IDE and simulator in Java, also usable as a cmdline emulator for a compiler tester.


The half float type - 16bit floating point, somewhat uncommon to find, but present in OpenCL. Conversion routines between float and half here, and here and here.

Mirror Mirror

Embedding a Flash MP3 player in web pages is nifty. Yahoo also. Tips on the use of mercurial. Free web hosting on bit-bucket, via hg. Some actual content on the web. SQL in your web browser.

Enjoy the Silence

Folklore about computers. Guess they've been around long enough.

OSX continues to surprise with little command line tools that wrap the features of the underlying frameworks e.g. sips - scriptable image processing system. Useful in combination with pdftk.

Online PDF and MP3 of language learning courses.

Starting Over

The site quirksmode.org has a great overview of JavaScript best practices. Makes dealing with cookies pretty painless and easy.

Always good to get more usable icons.

Python has built in templates, good for a start before bothering with a full blown template engine.

A Cry from the Crypt

Breaking into a running python program using pdb and Unix signals.

2d vectors in Python.

Learning Swedish. More learning Swedish, with online mp3 files. Mercurial hosting on bitbucket.

Cool tutorial on 2D games.

OpenGL Red book online. Basic introduction to vectors.

Computer generated imagery.

Open source OCR software, and papers about that. Cloth simulation. Again. Processing physics. GCD. Flixel.

Next in Line

The stdbuf program is now in GNU coreutils. Trying to handle buffered stdout in programs controlled via pipes is awkward.

Forget to Remember

A stylish introduction to the novelties of HTML5.

General collection of programming related 'stuff' - docforge looks interesting.

The Django framework in Python is quite nice, but needs, like all dynamic languages, a good way to debug for typos. The scratch module looks to be the simplest possible way to write a non DB backed web service, albeit with absolutely no support for safe concurrent data access as designed.

Planning a trip to MARS in a Cell. To read about Cell Execution and performance on Cell.

The HIPR Image processing textbook online.