Building StarLife for Mac and Linux

[tiagocc0]

Okay, so I started to build the prototype for Mac and Linux.
On Mac should work on 10.6 and up, on Linux I'm building it on Ubuntu 64 bits for now.

It successfully compiled on Mac already, I'm about to start compiling on Linux, however that is the easy part.
The hard part will be to bundle the libraries and necessary files together.

So I ask if anyone has experience deploying applications on Mac and/or Linux.

And also if anyone wants to test my builds when they are ready.

I will post more info about building on those platforms as I learn it.

 

[SCO]

Debian apt-get structure is nice. Launchpad is great to make both releases and daily builds too.

 

[Hirato]

It really depends on how you tackle your Linux builds.
As you're building for Ubuntu, Ideally you'd package up as a .deb and add the libraries you need as dependencies for the package, and so the user would install them with the package.

If you want to bundle the libraries, you basically have two ways of going about it
1) Link statically, AFAIK, passing the compiler -static should do it, I'd advise you to do more research.
2) Throw the needed libraries inside a bundled lib64/ folder (so the user can just get rid of them if that's what he wants), you can use "ldd" on the compiled binary to get a list of libraries loaded by an executable. You can also generally assume the user would have Qt, GTK, SDL and their dependents. And if you bundle anything that is linked against the likes of libpng and libjpeg, for the love of god, bundle those too.

 

[tiagocc0]

Debian apt-get structure is nice. Launchpad is great to make both releases and daily builds too.

Great! I will look into it, thanks.

It really depends on how you tackle your Linux builds.
As you're building for Ubuntu, Ideally you'd package up as a .deb and add the libraries you need as dependencies for the package, and so the user would install them with the package.

If you want to bundle the libraries, you basically have two ways of going about it
1) Link statically, AFAIK, passing the compiler -static should do it, I'd advise you to do more research.
2) Throw the needed libraries inside a bundled lib64/ folder (so the user can just get rid of them if that's what he wants), you can use "ldd" on the compiled binary to get a list of libraries loaded by an executable. You can also generally assume the user would have Qt, GTK, SDL and their dependents. And if you bundle anything that is linked against the likes of libpng and libjpeg, for the love of god, bundle those too.

Thanks for the info. I will probably add the libraries in a folder just like you said.

EDIT:
Compiled on Mac, had change just some paths, libvlc is not working yet on Mac.
Haven't bundled it yet, tried but gave me some strange errors.
Some parts of the program crashes.

 

[SCO]

The way it's normally done in ubuntu is to create 2 launchpad bazaar branches. One holds the program.
The other holds the debian files (strucuture for building and creating the package).

Then they are joined together into a 'recipe' which is used by the buildbot to build debs. If a library is not packaged for debian/ubuntu, you can link another 'repository' to the repository that builds that library building the recipe, so it has the library.

It's not a 'complete' solution that works for all O.S. (or even all linuxes, god-dammed redhat) but it's the standard apt-get structure on debian derivatives. Launchpad (the ubuntu cloud for this) has the advantage of giving you a buildbot for 'free'.
You can keep whatever build structure you use on other O.S. either on a branch somewhere or on the main repo; and only 'do' the debian folder for that branch on launchpad for debian derivatives installs so it's not either/or.

You get into trouble with this if you have a need for lots and lots of non-standard libraries obviously.

 

[tiagocc0]

It looks complicated as hell, but I guess I need to do it at least one time to understand it.

 

[SCO]

There are 2 things to understand:
The classical debian packaging; using debhelper or similar:
This can be learned in many web pages; but the best way is probably to pull some project packaging.
For instance do:
sudo apt-get install bzr
bzr co lp:~exult-team/exult-engine/packaging example

(this is a project i've done to make a daily build for exult, it will pull a debian folder where you can research the file names and file structure to see what's happening - the really important files are 'control', 'exult.install' and 'rules' but all the others tweak some aspects of the install or even the program itself - the 'patches' folder)

The launchpad infrastructure:
https://help.launchpad.net/

 

[tiagocc0]

Many thanks!

 

[SCO]

BTW, once you create a launchpad private key (you'll need it to create branches there) don't lose it! Backup that shit right away on some external usb or even your online email/gmail. I'm pretty sure they have no way to recover it since it would defeat the point of having a *private*-public key pair.

That key identifies the code you upload uniquely (it's signed). This is done so if someone does any monkey business on some code uploaded to launchpad there is a clear guilt indication; so that key is *valuable* even if you're not doing anything much important. Anyone malicious having it can fuck you, even if only to the point of making them delete your repositories (and possibly prosecution if a court can link you to it).

 

[SCO]

Also launchpad does automatic mirrors from offsite repositories.
You don't have to *change* your workflow to create launchpad builds if you use git or svn, just create a mirror (see how to here)

 

[tiagocc0]

That's pretty harsh, I will keep this in mind.
I just finished compiling in Linux, I need to check the libvlc library and then I can start to package it

 

[tiagocc0]

Tip: Always check file and dir names if it is the same as in the code.
Got a bunch of errors on Mac because of upper/lower characters that weren't different on the code for filenames.
Got a bunch more in Linux because of dirs too.

Tip2: Had a function to return random integers between min and max variables.
Worked like a charm in Windows but on Mac it would return a value between min and max that would sometimes turn out to be negative. EDIT: On Linux I had to do the same fix.

Errors so far:
1.Have to take a look at libvlc for linux and mac.
2.My parallax is not working on Mac yet, everything else looks ok.
3.Compiled on Linux, but it is giving me a realloc() error, like I'm trying to free memory of something that I shouldn't, funny is that it's the same code for mac and windows.. back to the board.

Solution for number 3:
Had to delete a qt 4 library that came with ubuntu to run the game.

 

[SCO]

If the qt4 library that you use can be statically linked you can use it like that (putting it on the 'Build-depends' of the source package instead of the 'Depends' of the program package on the control file) and it won't interfere with the newer ubuntu version.
Of course, downloads will be larger then, if a program is already using the lib and you'll have to muck around with the makefiles/cmakefiles/whatever to statically link

 

[tiagocc0]

Yeah, I'm seriously considering statically linking everything now.
I had some bad experience with it on Qt4 when I discovered that some modules couldn't be statically linked. I will look into it.

EDIT: Surprising News: It is damn fast on Ubuntu 12.10 64 bits inside a VM. I dare say a bit faster than running on Windows 8 (without VM).
Can't really tell the speed on Mac since it's an old machine and I using it with teamviewer.

EDIT2: Shader animation on Linux isn't quite the same for Windows. dang
Note: Have to change source of Qt5 in Linux and Mac to drop AntiAliasing for fonts in QML. dang dang

EDIT3: Fullscreen on Linux looks really weird.

 

[SCO]

BTW, you probably want to apply create a gmail filter for stuff coming from the launchpad buildbots. My filter for launchpad does this:

Matches: from: (noreply@launchpad.net | no_reply@launchpad.net | archive@ubuntu.com)
Do this: Skip Inbox, Apply label "Receipts/Launchpad"

Otherwise my inbox would be drowning in thousands of buildbot/upstream mirror commits crap emails.

I have some others for bugs and stuff on various bug databases. It's useful.

 

[tiagocc0]

BTW, you probably want to apply create a gmail filter for stuff coming from the launchpad buildbots. My filter for launchpad does this:

Matches: from: (noreply@launchpad.net | no_reply@launchpad.net | archive@ubuntu.com)
Do this: Skip Inbox, Apply label "Receipts/Launchpad"

Otherwise my inbox would be drowning in thousands of buildbot/upstream mirror commits crap emails.

I have some others for bugs and stuff on various bug databases. It's useful.

Thanks for the tip, I still start to package it on Linux first and worry about Mac later, I'm reading everything I can about it for now (mostly the links and tips you said above )

 

[tiagocc0]

While trying to build qt static I went ahead with the mac version.
Had to use this fix: http://stackoverflow.com/questions/516200/relative-paths-not-working-in-xcode-c
Then I had to compile a new macdeployqt from here: https://codereview.qt-project.org/#change,47906
Then I had to copy a bunch of libraries from qt5.0.2/5.0.2/clang64/qml (just copied everything within to the macos folder inside the .app bundle that qt makes automatically after compiling the game.

Now I'm using macdeployqt again to make a dmg file, I will test it on a vm with mac os x 10.8

 

[tiagocc0]

hmm, it's very confusing in Linux, should I put the executable in /bin, libs in /lib, where goes the game data?

 

[tuluse]

hmm, it's very confusing in Linux, should I put the executable in /bin, libs in /lib, where goes the game data?

I think it should be in /usr/bin and /usr/lib. Static game data could also go in /usr/lib, just make a star life directory. Actually, I just noticed debian and ubuntu provide a /usr/games, so you could use that too. User changeable game data like save files should go in /home/username.

 

[tiagocc0]

Thanks!

 

[tiagocc0]

Something very important for Qt users on Linux, I wasn't able to find much information on the official documentation, I guess it is scattered across several documents.
http://www.qtcentre.org/threads/40128-How-to-set-rpath-in-Qt-Project

With this I can put Qt shared libraries anywhere I want.
Didn't have much success with the static build because it seems QML doesn't like it.

 

[tiagocc0]

I think I can put everything in games and skip the bin folder, since there are executable files in the games folder, I will make a dir there though.

Finally sorted everything out for Linux too, now just need to create a .deb file and then launchpad

 

[SCO]

The 'starlife.install' file on the debian dir you'll have is what places files on the file tree. It whitelists and moves files from the build dir into the root dir.

If you look at the rules files, you'll see that i invoke configure with some standard 'paths'
dh_auto_configure -- --prefix=/usr \
--bindir=\$${prefix}/games \ <-exec from 'root' of the build dir to $${prefix}/games eg: usr/games
--mandir=\$${prefix}/share/man \ <-manual dir
--datadir=\$${prefix}/share/games \ <-exult specific makefile data dir
--with-timidity=\$${prefix}/share/games/exult \ <-exult specific makefile config file dir
This is the reason i can just use only one path without spaces in the following first 6 lines, they're already in the 'right' place to be copied over to root. In fact a deb file install is just that, copying over a subtree to the right places on the root + some sanity checks.

If your project needs some data in some 'directory' that is fixed, you obviously want a compile time parameter for it to support linux/MacOS installs, where the program and program data can be in wildly different places.

So taking the exult example, my exult.install:

usr/games/exult <- exe
usr/share/man/man6/exult.6 <-manual
usr/share/games/exult/*.flx <- exult data, wildcard for all flx files obviously
usr/share/games/exult/exultmsg.txt <- no idea
usr/share/games/exult/u7bgintrinsics.data <-exult data
usr/share/games/exult/u7siintrinsics.data <-exult data


These files aren't in the right place for ubuntu on the compile dir so they need to be moved. Mostly for files that aren't recognized by debhelper like icons or install scripts

usr/share/games/icons/exult.png usr/share/pixmaps
usr/share/games/applications/exult.desktop usr/share/applications

../../debian/gamedat-setup usr/share/games/exult
../../debian/timidity.cfg usr/share/games/exult

These are the keyring and sifixes files, which are mods for ultima 7 pt1 & pt2. They're produced by the exult makefile, but don't have a --moddir config parameter so need to be copied over manually.

content/bgkeyring/Keyring.cfg usr/share/games/exult/blackgate/mods
content/bgkeyring/Readme.txt usr/share/games/exult/blackgate/mods/bgkeyring
content/bgkeyring/data/* usr/share/games/exult/blackgate/mods/bgkeyring/data

content/sifixes/sifixes.cfg usr/share/games/exult/serpentisle/mods
content/sifixes/Readme.txt usr/share/games/exult/serpentisle/mods/sifixes
content/sifixes/data/* usr/share/games/exult/serpentisle/mods/sifixes/data

The 'exult.docs' lists files that should be placed unconditionally on the system docs folder for the app (i'm not quite sure if i shouldn't have placed the man page there). It's separated because doc files have their own permissions.

The postinst(all) and postrem(oval) are special script files that will be executed 'post-install' and 'post-removal'. I use them in this package to ask for the location of the games original data on the user hardrive, if needed (so they can be copied over to be usable by everyone), and on 'post-removal' to check if we are purging the program (a apt-get option meaning 'uninstall but eliminate config files too') to remove these original data files.

 

[tiagocc0]

Thanks SCO, your explanation really helped!
Finally managed to get a deb file, here's what I have done:
I created a usr folder, then a games, then StarLife and put everything I needed there.
Then I moved it to a 'Package' folder just to be safe
Package/usr/games/StarLife

then I compress it:

cd Package
tar -zcf starlife_0.71.orig.tar.gz usr

then I made a starlife-0.71 folder

cd starlife-0.71
dh_make -s --indep --createorig
echo "1.0" > debian/source/format
rm debian/*.ex
rm debian/*.EX

then I edited the file control and install, then finally

export LD_LIBRARY_PATH="/usr/games/StarLife/"
dpkg-buildpackage -rfakeroot -b -d -us -uc

But when I try to install I get a bunch of warnings:

Lintian check results for /home/tiagocc0/Desktop/starlife_0.71-1_all.deb:
E: starlife: arch-independent-package-contains-binary-or-object usr/games/StarLife/Qt/labs/folderlistmodel/libqmlfolderlistmodelplugin.so

however it does install and works

EDIT: Sources:
http://askubuntu.com/questions/27715/create-a-deb-package-from-scripts-or-binaries
http://forums.debian.net/viewtopic.php?t=35697
http://www.debian.org/doc/debian-policy/ch-controlfields.html

 

[SCO]

That E(rror) will prevent launchpad from building the package if you want to use it's buildbot. It's complaining that you've specified your package as architecture independent (you haven't specified it's i386 or x64 only so the source is assumed to be build-able on both) - you have two options to remove it, the best one; finding if there is a ubuntu library that contains libqmlfolderlistmodelplugin.so (if you are trying to link statically that so and put it on the source package, be advised that on ubuntu and debian generally, the libXXXXXX-dev packages are often used for that purpose), and adding it as a build-dependency (if static), or of a runtime dependency (if not static).

Taking the exult example again, in the control file on the Source section:
Build-Depends: debhelper (>= 7.0.50), po-debconf, autotools-dev, automake, libtool, libsdl1.2-dev, libsdl-mixer1.2-dev, byacc, flex, bison, libgimp2.0-dev, libglade2-dev, libfluidsynth-dev, libmt32emu-dev

I'd expect adding a 'libqt-dev' dependency to the Build-Depends line would allow static linking if the library permits it and not cause problems with other architectures. For instance, that's what i did with 'libmt32emu-dev' (which i built on another launchpad bot, but i wouldn't expect that to be needed for qt).

The other alternative is to hardcode the program to the architecture of that lib (in the Source field of the control file). Launchpad buildbot service only accepts source (edited out misleading info)

edit: actually i remembered: launchpad ONLY accepts sources packages, all of the binary packages in the archives have to be built on the launchpad cloud from a source package (that can be uploaded or constructed from a 'recipe' that pulls source from svn/git repositories) - but this doesn't mean you can't include the *.so for your build to link to, it's just bad policy since you need to specify the source architecture and it won't be available for others from the same source package.

BTW, if you want your program to be on the 'main' archives you have to go propose the package for inclusion (but most projects don't bother and depend on users doing 'apt-add-repository ppa:\whateverURI' instead). I never proposed a package for main, so i don't know what's the process but i'd expect you'd need to register on their mailing list and be the 'maintainer' of the thing (which would require checking if it keeps building with the latest system libraries probably)