Uploaded image for project: 'phpBB3'
  1. phpBB3
  2. PHPBB3-10188

Broken compressed output when errors/warnings are handled by phpbb and output_buffering is set to 4096 and phpbb gzip is enabled

    Details

    • Type: Bug
    • Status: Unverified Fix
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0.8
    • Fix Version/s: 3.0.9-RC1, 3.0.9-RC3
    • Component/s: Other
    • Labels:
      None

      Description

      Originally reported on Russian IST board: http://www.phpbbguru.net/community/topic16064-420.html#p326091

      If:

      1. output_buffering is set to 4096 in php.ini;
      2. gzip compression is enabled in phpbb;
      3. a warning or a notice is produced somewhere,

      then the resulting output is not a valid compressed stream, although it has gzip headers, and results in different browsers either reporting a compression error or just displaying nothing.

      In includes/functions.php line #3746 (current develop-olympus) there's a check:

      				// flush the content, else we get a white page if output buffering is on
      				if ((int) @ini_get('output_buffering') === 1 || strtolower(@ini_get('output_buffering')) === 'on')

      As per PHP manual on output_buffering setting:

      You can enable output buffering for all files by setting this directive to 'On'. If you wish to limit the size of the buffer to a certain size - you can use a maximum number of bytes instead of 'On', as a value for this directive (e.g., output_buffering=4096). As of PHP 4.3.5, this directive is always Off in PHP-CLI.

      Thus, the value of output_buffering setting in switched on state can differ from 1 or on.
      This causes page can't be displayed on E_NOTICE or E_WARNING in some cases.

        Issue Links

          Activity

          Hide
          bantu Andreas Fischer added a comment -

          $output_buffering_on = (@ini_get('output_buffering') > 0 || strtolower(@ini_get('output_buffering')) === 'on');

          Show
          bantu Andreas Fischer added a comment - $output_buffering_on = (@ini_get('output_buffering') > 0 || strtolower(@ini_get('output_buffering')) === 'on');
          Hide
          bantu Andreas Fischer added a comment -

          Basically change "=== 1" to "> 0".

          Show
          bantu Andreas Fischer added a comment - Basically change "=== 1" to "> 0".
          Hide
          Oleg Oleg [X] (Inactive) added a comment -

          Reproduce code:

          (04:13:34) nn-: <?php echo 2; var_dump(ob_start("ob_gzhandler")); echo 1; exit; ob_start("ob_gzhandler"); exit;
          (04:13:39) nn-: => content encoding error
          (04:14:09) nn-: <?php echo 2; ob_flush(); var_dump(ob_start("ob_gzhandler")); echo 1; exit; ob_start("ob_gzhandler"); exit;
          (04:14:15) nn-: => 2bool(true) 1
          (04:14:43) nn-: this is with ob enabled in php.ini
          (04:15:35) nn-: flush not needed when ob turned off in php.ini

          This code must be executed via apache. I was unable to coerce php to do compression from command line. Because of this I can't write a test for this breakage.

          Even the following does not work:

          wooka% echo '<?php ob_start(); $_SERVER["GATEWAY_INTERFACE"]="CGI/1.1";$_SERVER["HTTP_PROTOCOL"]="HTTP/1.1";$_SERVER["HTTP_ACCEPT_ENCODING"]="gzip,deflate"; var_dump(ob_start("ob_gzhandler")); echo 1; ob_start("ob_gzhandler"); var_dump(ob_gzhandler("foobar",6)); exit; ' |php -d zlib.output_compression=on
          bool(true)
          1bool(false)

          There is nothing terribly obvious in the source for zlib extension other than the fact that it may not be reading $_SERVER. It almost feels like php maintains some sort of a shadow copy of headers that is used.

          On the topic of gzip compression creating unusable output:

          (04:16:00) nn-: so what happens is ob_start with ob_gzhandler does something similar to headers_sent check
          (04:16:12) nn-: and because output buffering is on headers were not sent
          (04:16:20) nn-: however all output eventually is sent out including uncompressed output
          (04:16:24) nn-: which ruins the compressed stream
          (04:16:26) nn-: php fail
          (04:17:03) nn-: of course, technically ob may have been enabled to collect output
          (04:17:28) nn-: so this essentially is an unresolvable problem as far as php is concerned
          (04:17:37) nn-: php architecture fail
          (04:17:58) nn-: news at 11
          (04:19:08) nn-: no actually i think this is detectable
          (04:19:29) nn-: when gzhandler is about to set headers, which should be right before it starts output, it should check if any output has been sent
          (04:20:28) nn-: of course gzip supports concatenating compressed streams
          (04:20:56) nn-: so technically i could do ob_start, collect output, compress it, echo it
          (04:21:08) nn-: then have some more output passing through gzhandler
          (04:21:19) nn-: and some user agents may be able to understand all of it
          (04:22:34) nn-: the concept of a single output stream makes life easier for newbies and causes endless problems for everyone else

          I added a test for detecting whether output buffering is enabled, I cannot figure out how to test for broken compression.

          Show
          Oleg Oleg [X] (Inactive) added a comment - Reproduce code: (04:13:34) nn-: <?php echo 2; var_dump(ob_start("ob_gzhandler")); echo 1; exit; ob_start("ob_gzhandler"); exit; (04:13:39) nn-: => content encoding error (04:14:09) nn-: <?php echo 2; ob_flush(); var_dump(ob_start("ob_gzhandler")); echo 1; exit; ob_start("ob_gzhandler"); exit; (04:14:15) nn-: => 2bool(true) 1 (04:14:43) nn-: this is with ob enabled in php.ini (04:15:35) nn-: flush not needed when ob turned off in php.ini This code must be executed via apache. I was unable to coerce php to do compression from command line. Because of this I can't write a test for this breakage. Even the following does not work: wooka% echo '<?php ob_start(); $_SERVER ["GATEWAY_INTERFACE"] ="CGI/1.1";$_SERVER ["HTTP_PROTOCOL"] ="HTTP/1.1";$_SERVER ["HTTP_ACCEPT_ENCODING"] ="gzip,deflate"; var_dump(ob_start("ob_gzhandler")); echo 1; ob_start("ob_gzhandler"); var_dump(ob_gzhandler("foobar",6)); exit; ' |php -d zlib.output_compression=on bool(true) 1bool(false) There is nothing terribly obvious in the source for zlib extension other than the fact that it may not be reading $_SERVER. It almost feels like php maintains some sort of a shadow copy of headers that is used. On the topic of gzip compression creating unusable output: (04:16:00) nn-: so what happens is ob_start with ob_gzhandler does something similar to headers_sent check (04:16:12) nn-: and because output buffering is on headers were not sent (04:16:20) nn-: however all output eventually is sent out including uncompressed output (04:16:24) nn-: which ruins the compressed stream (04:16:26) nn-: php fail (04:17:03) nn-: of course, technically ob may have been enabled to collect output (04:17:28) nn-: so this essentially is an unresolvable problem as far as php is concerned (04:17:37) nn-: php architecture fail (04:17:58) nn-: news at 11 (04:19:08) nn-: no actually i think this is detectable (04:19:29) nn-: when gzhandler is about to set headers, which should be right before it starts output, it should check if any output has been sent (04:20:28) nn-: of course gzip supports concatenating compressed streams (04:20:56) nn-: so technically i could do ob_start, collect output, compress it, echo it (04:21:08) nn-: then have some more output passing through gzhandler (04:21:19) nn-: and some user agents may be able to understand all of it (04:22:34) nn-: the concept of a single output stream makes life easier for newbies and causes endless problems for everyone else I added a test for detecting whether output buffering is enabled, I cannot figure out how to test for broken compression.
          Show
          Oleg Oleg [X] (Inactive) added a comment - Untested: https://github.com/p/phpbb3/compare/ticket%2F10188
          Hide
          rxu Ruslan Uzdenov added a comment -

          The pull request #1969 has been cleaned. nn- has came up with the different fix: https://github.com/p/phpbb3/compare/ticket%2F10188

          Show
          rxu Ruslan Uzdenov added a comment - The pull request #1969 has been cleaned. nn- has came up with the different fix: https://github.com/p/phpbb3/compare/ticket%2F10188
          Hide
          naderman Nils Adermann added a comment - - edited

          When a non-fatal error occurs at the beginning of the script before any custom
          error handler is set one of two situations can be encountered:

          • 1) if the ini option output buffer is disabled:
            • headers are sent to the http client
            • the error message is output
          • 2) if the ini option output_buffer is enabled or the script
            is run within an ob_start()/ob_end() wrapper:
            • the error message is written to the output buffer

          Once the script reaches page_header() phpbb starts gzip compression if enabled.
          This is done through ob_start with a ob_gzhandler as a callback. The
          compression is skipped if headers have already been sent. In situation 1) the
          error message sent in plain text comes with headers and this gzip compression
          is skipped. The client receives a plaintext version of the page. However in
          situation 2) headers have not been sent yet and the rest of the page will be
          compressed. The result is a plaintext error message followed by compressed
          output. The client does not understand this output resulting in either an
          error message or simply a blank page in the browser.

          In addition to the above situation this problem occurs with errors that are
          triggered after the custom error handler is loaded. The problem has been
          noticed before, and a workaround was found. The error handler would call
          ob_flush() for particular configuration settings before outputting the error
          message. This resulted in headers being sent when output buffering was enabled
          thus disabling gzip compression for the rest of the page. The constraints under
          which ob_flush() was called were lessened over time whenever a new case was
          found that would trigger this problem. Eventually ob_flush() would be called
          even when code causing an E_NOTICE was simply run within an ob_start/ob_end.
          This makes it impossible to use output buffering to retrieve the content of an
          error message without prohibiting the page from setting headers afterwards.

          This commit removes all flushing in msg_handler completely and instead fixes
          the problem for both errors before and after the error handler is registered.
          GZIP compression is only enabled if there is at most one level of output
          buffering (e.g. the output_buffer php.ini option is enabled) and if there has
          not yet been any output in this buffer. This should avoid any partial output
          compression.

          Show
          naderman Nils Adermann added a comment - - edited When a non-fatal error occurs at the beginning of the script before any custom error handler is set one of two situations can be encountered: 1) if the ini option output buffer is disabled: headers are sent to the http client the error message is output 2) if the ini option output_buffer is enabled or the script is run within an ob_start()/ob_end() wrapper: the error message is written to the output buffer Once the script reaches page_header() phpbb starts gzip compression if enabled. This is done through ob_start with a ob_gzhandler as a callback. The compression is skipped if headers have already been sent. In situation 1) the error message sent in plain text comes with headers and this gzip compression is skipped. The client receives a plaintext version of the page. However in situation 2) headers have not been sent yet and the rest of the page will be compressed. The result is a plaintext error message followed by compressed output. The client does not understand this output resulting in either an error message or simply a blank page in the browser. In addition to the above situation this problem occurs with errors that are triggered after the custom error handler is loaded. The problem has been noticed before, and a workaround was found. The error handler would call ob_flush() for particular configuration settings before outputting the error message. This resulted in headers being sent when output buffering was enabled thus disabling gzip compression for the rest of the page. The constraints under which ob_flush() was called were lessened over time whenever a new case was found that would trigger this problem. Eventually ob_flush() would be called even when code causing an E_NOTICE was simply run within an ob_start/ob_end. This makes it impossible to use output buffering to retrieve the content of an error message without prohibiting the page from setting headers afterwards. This commit removes all flushing in msg_handler completely and instead fixes the problem for both errors before and after the error handler is registered. GZIP compression is only enabled if there is at most one level of output buffering (e.g. the output_buffer php.ini option is enabled) and if there has not yet been any output in this buffer. This should avoid any partial output compression.
          Hide
          Oleg Oleg [X] (Inactive) added a comment -
          Show
          Oleg Oleg [X] (Inactive) added a comment - For posterity: https://gist.github.com/a5193d10756b1edc3f7e

            People

            • Assignee:
              naderman Nils Adermann
              Reporter:
              rxu Ruslan Uzdenov
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development