Currently the extensions autoloaders are loaded with load_extensions() in the container builder. The idea is that they will be cached using the ConfigCache. However, as we've noticed during testing this is not actually the case and every page results in a complete recompilation of a reduced container in the load_extensions() method due to that. As a result, every page load is quite a bit slower.