Updating B4 => B5 causes memory error

Trying to update a B4 to B5 causes to a update error.
This happens after the first diff, then I download the new files(I has now FTP on my server ), upload them, and choose check files again. After that, lighttpd is showing a 500 error, and in my errorlog I find this:

Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 71 bytes) in /var/www/phpbbservice.nl/www/install/update/new/includes/diff/engine.php on line 90

My memory limit is 32MB. After bumping it up to 64MB it works correct.