Deterministic builds of my programs

January 4th, 2015 No comments

It is actually quite easy to build my programs in a deterministic way, but there are a few caveats. The only reason this is important is to enable users (you) to be able to verify that I don’t modify the source code before I compile the binaries (e.g. inserting a virus).

I think the amount of effort to have more and more software built in a deterministic way will increase in the next few years. Since I have my TrueCrypt related projects, I stumbled upon this article where someone verifies the TrueCrypt Windows binaries, and this pushed me to finally investigate how to make sure my own programs are able to be verified.

These days I’m recommending people to use Cygwin to compile my programs. It just makes everything so much easier. So if you have the same version of all your cygwin packages as me, then our builds should produce the same results. For instance, here are the sha256 hashes of the binaries compiling the current HEAD of AltDrag (000baf8) with the latest cygwin mingw-w64 compilers:

$ ./build.bat all x64
$ (cd build && sha256sum AltDrag.exe hooks.dll HookWindows_x64.exe hooks_x64.dll)
56c6741c2074fedd12e2b9f1dac44a628cceecf176379b1b37bf43511030ea1e *AltDrag.exe
a6ae55bce191d0da5dc7cf4ffa73a5a33979ea6afa186dba31bac198efabea6d *hooks.dll
3b9dd364cc7e29d05cc2b45616285222521fe7ce2ffe708169c89f9fc489ee7c *HookWindows_x64.exe
e2a51731951863067c54a50bac08eae30acff7ec3ed80d88eed5ef790b705ab4 *hooks_x64.dll

You can go ahead and do this right now and you should get the same hashes. This doesn’t really prove that much though since these binaries are not part of any release, but having an easy procedure to do this is useful for the next release.

It should work fine if you have the same mingw-w64 packages as me, but the longer you wait after the release, it will become more and more difficult to double-check this. The reason is because Cygwin doesn’t keep a complete archive of old versions. I could provide a mirror of my current Cygwin packages, but that isn’t safe because I could have modified these packages in beforehand. If the Cygwin people would provide signatures for the packages, then this would be a valid approach. It would also be a good way to verify that a mirror is trustworthy. They only seem to provide md5 hashes, but when these disappear from the mirrors and there is no way to get these hashes from a trusted source, the point kinda disappears. md5 is also known to be broken and should not really be trusted for file integrity. It’s easy to get the list of the installed packages with `cygcheck -c`, but the list is kinda overwhelming and it’s not clear exactly what packages and libraries make a difference. If you would read this blog post shortly after I publish it however, hopefully no new packages have been published yet and you could just get the latest of the mingw-w64 packages and their dependencies, and it should work. This would also be true for new releases.

Doing this more properly is probably more important for key software that a lot of people use and depend on, e.g. cryptography software, Bitcoin, etc. For this kind of software, a new release probably would have to be verified by multiple trusted sources before being made available (perhaps even automatically made available once this criteria is satisfied, this security feature is something GitHub could implement). But for TrueCrypt, where the authors themselves shouldn’t be automatically trusted, it is more difficult. Luckily the person verifying TrueCrypt was able to get the exact compiler that they used (MSVC with the exact same patches applied). This would be more difficult if they used mingw that changes a lot more and older versions are not as easily reproduced.

There’s one more binary to verify, and that’s the installer. I’ve noticed that NSIS record the last modified time of the files it is including, so to get an exact match, you would have to make sure your files have the same timestamp as my files have. When the NSIS installer is extracting them, it applies the recorded timestamp on the files. Having an option to disable this in NSIS would be useful for this purpose (or set the timestamps manually for each file in the installer script). I attempted to write a patch to do this but I gave up when I failed to build it for testing. The simplest way seems to be to set build_datesave=0 in the source code.

In cygwin you can use the touch command to set a file’s last modified time. If all of the hashes above is correct (and make sure AltDrag.ini is the same that’s checked in to Git), and you set all of them to have the same timestamp as me before you build the installer, and you use NSIS Unicode 2.46.5 (with the AccessControl plug-in v1.0.8.1), then you should get the same hash as me.

$ touch -d '2015-01-03 23:37:00 UTC' AltDrag.ini build/{AltDrag.exe,hooks.dll,HookWindows_x64.exe,hooks_x64.dll} 
$ makensis installer.nsi
$ (cd build && sha256sum AltDrag-1.0.exe)
c2b4c066cde74c08d24b16d6ec77ef3cc1ec5e41fdb6938788dc49dd894ca211 *AltDrag-1.0.exe

It is difficult to get old versions of NSIS plugins, which sucks. NSIS plugins are saved inside the installer as dll files, similar to files that are about to be installed, but thankfully their last modified timestamp is not saved.

I guess that wraps it up for this blog post. I’d be thrilled to hear if anyone want to verify my releases in the future. An unexpected bonus this brings is that if my machine is infected with viruses, this would be detected when I make a release. I usually run virus checks on my computer before releasing anything important, though, just as a precaution.

P.S. Normally you would have to be worried about the PE header timestamp, but mingw-w64 sets it to some other values that I don’t really know the meaning of (it’s not zeroed). FYI the diff tool used in TrueCrypt comparison article is Beyond Compare, it is quite good but unfortunately payware (trial available).

2015 new year’s resolutions

January 1st, 2015 No comments
  • Only release reproducible / deterministic binaries.
  • Post checksums.
  • Sign binaries.

P.S. This website has supported https for a while now. I’ll probably enable HSTS in 2015.

Tags: ,

Installing TrueCrypt on Yosemite

October 28th, 2014 19 comments

I have a few projects around TrueCrypt but I’ve never written about them here on my blog, for instance I’ve contributed a lot to the truecrypt-archive project on GitHub, and I have a PPA for Ubuntu.

For my PPA, I wrote a patch to support Ubuntu’s appindicators, without which TrueCrypt is a lot harder to use on Ubuntu. I was updating the PPA this weekend for 14.10 and I took the opportunity to add bash completion as well. I recently discovered that the PPA is quite popular, which makes me happy.

Anyway, back to the point of this blog post. Mac OS X 10.10 (Yosemite) was recently released, and both TrueCrypt 7.1a and 7.2 are unable to install on it due to a version check. Instead of posting a new binary that people don’t trust, here are steps you can take to remove the installer check from your own binary. You need to know how to use the terminal to perform these steps.

The easy way is to first extract TrueCrypt 7.1a.mpkg from TrueCrypt 7.1a Mac OS X.dmg and then run this command in the terminal:

sed -i '' 's/<installation-check .*>//' 'TrueCrypt 7.1a.mpkg/Contents/distribution.dist'

Or, if you want to create a new dmg archive with this fix, you can follow these steps:

hdiutil convert 'TrueCrypt 7.1a Mac OS X.dmg' -format UDRW -o 'TrueCrypt 7.1a Mac OS X writable.dmg'
hdiutil attach 'TrueCrypt 7.1a Mac OS X writable.dmg'
sed -i '' 's/<installation-check .*>//' '/Volumes/TrueCrypt 7.1a/TrueCrypt 7.1a.mpkg/Contents/distribution.dist'
hdiutil detach '/Volumes/TrueCrypt 7.1a'
hdiutil convert 'TrueCrypt 7.1a Mac OS X writable.dmg' -format UDBZ -o 'TrueCrypt 7.1a Mac OS X Yosemite.dmg'

I hope this is useful to people.

Edit: See here for more of my TrueCrypt Mac patches: https://github.com/stefansundin/truecrypt-mac

Tags: , ,

ShutdownGuard featured in PC Welt magazine

May 10th, 2014 No comments

The German magazine PC Welt featured ShutdownGuard in edition 5/2014. Compared to when AltDrag was featured in another german magazine back in 2012, this time there’s good chunk of text in the pages! My only complaint is that they didn’t put my name in there, but otherwise I’m very happy!

What magazine is next?

Google Project Hosting is dead, long live GitHub

January 16th, 2014 No comments

Yesterday Google Code Project Hosting shut down the ability for projects to add new downloads, just as they announced back in May. The reason they say, is because the “downloads have become a source of abuse”. I have no doubt they were having problems, but this move — basically killing Project Hosting instead of fixing it — shows that they really see no future in the product. I had some trouble with this back in March and a rather unpleasant experience trying to get Google to correct it.

It was perhaps no coincidence that GitHub less than two months later announced “releases”, finally making it possible to ship binaries to end users (they did have a downloads section for a while, which has since been removed). With this launch, GitHub is offering a lot more than Google has ever offered with Project Hosting.

Last weekend I spent some time to finally migrate my repositories over to Git and GitHub. All my repositories on Google were using Subversion, the first VCS that they offered (they now also offer Mercurial and Git). It has been clear for a while now that Git has won the DCVS fight, so this feels very familiar to the days when people were moving from CVS to Subversion.

GitHub suggests that you use svn2git to migrate from Subversion to Git. The only issue I had with this is that it is impossible to use it to migrate your Project Hosting-specific /wiki directory, and I had a lot of trouble finding a good solution. Finally I stumbled over Google’s instructions on converting your repository from Subversion to Git. What I ended up using is a combination of the two.

You really only have one good option: import the /wiki directory as a separate branch.

You could also import your /wiki directory as a GitHub wiki, but they suck. Another option is to use git subtree to add your /wiki directory into your master branch, but I don’t recommend it.

First, use svn2git to transfer everything except your /wiki directory. Your /trunk will become your master branch, /branches will become their own branches and /tags will automatically be tagged. Be sure to build your ~/.svn2git/authors file first, and always run svn2git with the verbose flags, since otherwise you might not see important conversion errors. If your branches and tags are not correctly converted, watch out for this bug.

mkdir altdrag
cd altdrag
svn2git -v http://altdrag.googlecode.com/svn/

Then, use “git svn clone” to transfer your /wiki directory to its own little git repo. Use --no-metadata to get the commit messages in the same format as svn2git.

cd ..
git svn clone http://altdrag.googlecode.com/svn/wiki altdrag.wiki --authors-file=$HOME/.svn2git/authors --no-metadata
cd altdrag.wiki
git branch -m doc

I also decided to rename my tags from the traditional “altdrag-0.1” Subversion syntax to semantic versioning, commonly used with Git.

git tag v0.1 altdrag-0.1
git tag v0.2 altdrag-0.2
...
git tag -d altdrag-0.1 altdrag-0.2 ...

Now you should be all set to push your repository to GitHub. Create your repo on GitHub, then add it as a new remote and push everything.

git remote add origin git@github.com:stefansundin/altdrag.git
git push -u origin --all
git push --tags

Now all you have left to do is to move over your binaries to the release tags that were automatically created.

There is one thing that GitHub is still missing; download counters. GitHub is targeting the developer community, where the number of stars your repo has is your indicator of how popular your project is, but many of us also build programs intended for ordinary mortals who don’t have GitHub accounts. I suggest that you build a github.io page that is more user friendly that the GitHub repository page, and use a JavaScript-based analytic tool to track the number of downloads, and then try to funnel most of your users there (e.g. like this).

Update: GitHub actually has download counters, but the stats is only available through the API. E.g. search for “download_count” here: https://api.github.com/repos/stefansundin/altdrag/releases

I will still have to go back to Google Project Hosting from time to time during this transition. I still have lots of issue reports on Project Hosting that I intend to resolve there. And when I release the next version, I have to transfer the feature that checks if a new version has been released to GitHub.

It feels really good to finally move over to GitHub. Google Project Hosting was fun while it lasted, but I hereby pronounce it dead.