Framework_ConstraintTest::testConstraintIsEqual2 fails with data set #1

[sebastianbergmann]

This test fails on my machine with both PHP 5.4 and PHP 5.5 while it passes on Travis CI.

➜  phpunit git:(master) /usr/local/php-5.4/bin/php phpunit.php --stop-on-failure
PHPUnit 3.8-g899e18f by Sebastian Bergmann.

Configuration read from /usr/local/src/phpunit/phpunit.xml.dist

.............................................................   61 / 1332 (  4%)
.............................................................  122 / 1332 (  9%)
.............................................................  183 / 1332 ( 13%)
.............................................................  244 / 1332 ( 18%)
.............................................................  305 / 1332 ( 22%)
.............................................................  366 / 1332 ( 27%)
.............................................................  427 / 1332 ( 32%)
.............................................................  488 / 1332 ( 36%)
.............................................................  549 / 1332 ( 41%)
.............................................................  610 / 1332 ( 45%)
.............................................................  671 / 1332 ( 50%)
.............................................................  732 / 1332 ( 54%)
................F

Time: 15.31 seconds, Memory: 48.25Mb

There was 1 failure:

1) Framework_ConstraintTest::testConstraintIsEqual2 with data set #1 (1.1000000000000001, 0, 'Failed asserting that 0 matches expected 1.1.
')
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
 'custom message
-Failed asserting that 0 matches expected 1.1.
+Failed asserting that 0 matches expected 1.1000000000000001.
 '

/usr/local/src/phpunit/PHPUnit/Framework/Constraint/IsEqual.php:172
/usr/local/src/phpunit/PHPUnit/Framework/Assert.php:2164
/usr/local/src/phpunit/PHPUnit/Framework/Assert.php:481
/usr/local/src/phpunit/Tests/Framework/ConstraintTest.php:956
/usr/local/src/phpunit/PHPUnit/Framework/TestCase.php:987
/usr/local/src/phpunit/PHPUnit/Framework/TestCase.php:839
/usr/local/src/phpunit/PHPUnit/Framework/TestResult.php:661
/usr/local/src/phpunit/PHPUnit/Framework/TestCase.php:782
/usr/local/src/phpunit/PHPUnit/Framework/TestSuite.php:729
/usr/local/src/phpunit/PHPUnit/Framework/TestSuite.php:729
/usr/local/src/phpunit/PHPUnit/Framework/TestSuite.php:729
/usr/local/src/phpunit/PHPUnit/Framework/TestSuite.php:729
/usr/local/src/phpunit/PHPUnit/TextUI/TestRunner.php:415
/usr/local/src/phpunit/PHPUnit/TextUI/Command.php:173
/usr/local/src/phpunit/PHPUnit/TextUI/Command.php:126

FAILURES!
Tests: 749, Assertions: 1057, Failures: 1.

Generating code coverage report in Clover XML format ... done

Generating code coverage report in HTML format ... done


➜  phpunit git:(master) /usr/local/php-5.5/bin/php phpunit.php --stop-on-failure
PHPUnit 3.8-g899e18f by Sebastian Bergmann.

Configuration read from /usr/local/src/phpunit/phpunit.xml.dist

.............................................................   61 / 1332 (  4%)
.............................................................  122 / 1332 (  9%)
.............................................................  183 / 1332 ( 13%)
.............................................................  244 / 1332 ( 18%)
.............................................................  305 / 1332 ( 22%)
.............................................................  366 / 1332 ( 27%)
.............................................................  427 / 1332 ( 32%)
.............................................................  488 / 1332 ( 36%)
.............................................................  549 / 1332 ( 41%)
.............................................................  610 / 1332 ( 45%)
.............................................................  671 / 1332 ( 50%)
.............................................................  732 / 1332 ( 54%)
................F

Time: 14.59 seconds, Memory: 48.25Mb

There was 1 failure:

1) Framework_ConstraintTest::testConstraintIsEqual2 with data set #1 (1.1000000000000001, 0, 'Failed asserting that 0 matches expected 1.1.
')
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
 'custom message
-Failed asserting that 0 matches expected 1.1.
+Failed asserting that 0 matches expected 1.1000000000000001.
 '

/usr/local/src/phpunit/PHPUnit/Framework/Constraint/IsEqual.php:172
/usr/local/src/phpunit/PHPUnit/Framework/Assert.php:2164
/usr/local/src/phpunit/PHPUnit/Framework/Assert.php:481
/usr/local/src/phpunit/Tests/Framework/ConstraintTest.php:956
/usr/local/src/phpunit/PHPUnit/Framework/TestCase.php:987
/usr/local/src/phpunit/PHPUnit/Framework/TestCase.php:839
/usr/local/src/phpunit/PHPUnit/Framework/TestResult.php:661
/usr/local/src/phpunit/PHPUnit/Framework/TestCase.php:782
/usr/local/src/phpunit/PHPUnit/Framework/TestSuite.php:729
/usr/local/src/phpunit/PHPUnit/Framework/TestSuite.php:729
/usr/local/src/phpunit/PHPUnit/Framework/TestSuite.php:729
/usr/local/src/phpunit/PHPUnit/Framework/TestSuite.php:729
/usr/local/src/phpunit/PHPUnit/TextUI/TestRunner.php:415
/usr/local/src/phpunit/PHPUnit/TextUI/Command.php:173
/usr/local/src/phpunit/PHPUnit/TextUI/Command.php:126

FAILURES!
Tests: 749, Assertions: 1057, Failures: 1.

Generating code coverage report in Clover XML format ... done

Generating code coverage report in HTML format ... done

[whatthejeff]

Looks like your precision is configured higher than 16.

$ php -a
Interactive shell

php > print ini_get('precision') . "\n";
14
php > print var_export(1.1, true) . "\n";
1.1
php > ini_set('precision', 17);
php > print var_export(1.1, true) . "\n";
1.1000000000000001

Maybe we should explicitly set the precision in the bootstrap/configuration file?

[sebastianbergmann]

I am using PHP's defaults as my php.ini does not set precision or serialize_precision.

➜  phpunit git:(master) php -i|grep precision
precision => 14 => 14
serialize_precision => 17 => 17

[whatthejeff]

That's the only way I can think that 1.1 is going to be displayed as 1.1000000000000001 (outside of using --process-isolation since serialization has a precision higher than 16).

Could something be changing your precision at runtime? Would it be possible for you to add some debugging to that test that records the output from ini_get('precision')?

[whatthejeff]

The precision of that number is 17, right? Unless I'm misunderstanding the point of the precision configuration, I don't even understand how you could get that number if your precision weren't 17. For instance:

php > print ini_get('precision') . "\n";
14
php > print var_export(1.1000000000000001, true) . "\n";
1.1
php > ini_set('precision', 17);
php > print var_export(1.1000000000000001, true) . "\n";
1.1000000000000001

[sebastianbergmann]

PHP 5.5.6-dev (cli) (built: Nov  8 2013 10:40:44) (DEBUG)
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2013 Zend Technologies
    with Xdebug v2.3.0dev, Copyright (c) 2002-2013, by Derick Rethans


php > print ini_get('precision') . "\n";
14
php > print var_export(1.1000000000000001, true) . "\n";
1.1000000000000001
php > ini_set('precision', 17);
php > print var_export(1.1000000000000001, true) . "\n";
1.1000000000000001

[whatthejeff]

Oh, i think I found out what is going on here. Seems var_export was updated to use the serialize_precision configuration instead of the precision configuration 10 days ago.

See:

• https://bugs.php.net/bug.php?id=64760
• php/php-src@4dc4302

[whatthejeff]

Does setting serialize_precision to 14 fix the problem for you?

[sebastianbergmann]

Yes, otherwise I would not have committed c1fe4b7 ;-)

[whatthejeff]

Oh, okay. I guess we can close this ticket now. Curious to see if this breaks any other libs.