!!! TASK: Use integers as lightweight nodeanchorpoints

[kitsunet]

The nodeanchorpoints are an implementation detail of the DBAL adapter, using integers in the projection saves space and helps improving query performance.

The change is surprisingly simple. Due to the locking mechanism it should be fairly safe to do this with auto increments even under load.

Important

This change requires a CR replay if you already have an installation with data.
Note that resetall will only be available via #4864. Previously you can use cr:replayall and ignore the error ;)

flow cr:setup --content-repository default --reset-projections
flow cr:replayall --content-repository default

[dlubitz]

Looks good in general.

I just raised a question about the need of NodeRelationAnchorPoint::create. And I wonder how the migration path looks like? I guess replay? If yes, we should document this in the PR description.

[mhsdesign]

In my testings i had a little trouble while migrating the database.

I assumed that those are the steps:

flow cr:setup --content-repository default
flow cr:replayall --content-repository default

but a first (and any directly following subsequent run of cr:setup) will result in an error:

Error

flow cr:setup --content-repository twodimensions
An exception occurred while executing 'ALTER TABLE cr_twodimensions_p_graph_node CHANGE relationanchorpoint relationanchorpoint BIGINT AUTO_INCREMENT NOT NULL, CHANGE classification classification VARCHAR(20) CHARACTER SET binary NOT NULL':
SQLSTATE[22007]: Invalid datetime format: 1292 Truncated incorrect
INTEGER value: '02c1bcbf-eb33-4e7c-b2d2-4e4a4c332272'

  Type: Doctrine\DBAL\Exception\DriverException
  File: Packages/Libraries/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriv
        er.php
  Line: 128

Nested exception:
SQLSTATE[22007]: Invalid datetime format: 1292 Truncated incorrect INTEGER value: '02c1bcbf-eb33-4e7c-b2d2-4e4a4c332272'

  Type: Doctrine\DBAL\Driver\PDO\Exception
  Code: 22007
  File: Packages/Libraries/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDO/Exception.php
  Line: 18

Nested exception:
SQLSTATE[22007]: Invalid datetime format: 1292 Truncated incorrect INTEGER value: '02c1bcbf-eb33-4e7c-b2d2-4e4a4c332272'

  Type: PDOException
  Code: 22007
  File: Packages/Libraries/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php
  Line: 55

then i tried out of fun the replayall directly afterwards which also fails:

Error

flow cr:replayall --content-repository twodimensions
Replaying events for all projections of Content Repository "twodimensions" ...
Failed to update and commit highest applied sequence number for subscriber "Neos\ContentGraph\DoctrineDbalAdapter\DoctrineDbalContentGraphProjection".
Please run
Neos\EventStore\DoctrineAdapter\DoctrineCheckpointStorage::setup()

  Type: Neos\EventStore\Exception\CheckpointException
  Code: 1652279375
  File: Packages/Libraries/neos/eventstore-doctrineadapter/src/DoctrineCheckpointSt
        orage.php
  Line: 80

Nested exception:
Transaction commit failed because the transaction has been marked for rollback only.

  Type: Doctrine\DBAL\ConnectionException
  File: Packages/Libraries/doctrine/dbal/lib/Doctrine/DBAL/ConnectionException.php
  Line: 15

Nested exception:
An exception occurred while executing 'INSERT INTO cr_twodimensions_p_graph_node (nodeaggregateid, origindimensionspacepoint, origindimensionspacepointhash, properties, nodetypename, classification, created, originalcreated, lastmodified, originallastmodified) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params ["2fbc3faf-663a-4ef0-b586-7b2c1c7fd342", "[]", "d751713988987e9331980363e24189ce", "[]", "Neos.Neos:Sites", "root", "2024-01-14 13:33:11", "2024-01-14 13:33:11", null, null]:
SQLSTATE[HY000]: General error: 1364 Field 'relationanchorpoint' doesn't
have a default value

  Type: Doctrine\DBAL\Exception\NotNullConstraintViolationException
  File: Packages/Libraries/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriv
        er.php
  Line: 125

Nested exception:
SQLSTATE[HY000]: General error: 1364 Field 'relationanchorpoint' doesn't have a default value

  Type: Doctrine\DBAL\Driver\PDO\Exception
  Code: HY000
  File: Packages/Libraries/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDO/Exception.php
  Line: 18

Nested exception:
SQLSTATE[HY000]: General error: 1364 Field 'relationanchorpoint' doesn't have a default value

  Type: PDOException
  Code: HY000
  File: Packages/Libraries/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php
  Line: 117

I know that the definition insanity is doing the same thing over and over and expecting different results, but as a programmer i had to try - and indeed. Now both commands pass:

Neos git:(9.0) ✗ flow cr:setup --content-repository twodimensions    
Content Repository "twodimensions" was set up
Neos git:(9.0) ✗ flow cr:replayall --content-repository twodimensions
Replaying events for all projections of Content Repository "twodimensions" ...
Done.

---

but besides this i can confirm that this change works with the neos ui and demo and that the neos ui e2e tests are working locally!

[kitsunet]

Right, mmm the migration, not too much we can do about it, the values obviously do not fit at all in the new schema if you have entries already....

[mhsdesign]

yes but i was confused that this order of commands results in a migrated state:

1. flow cr:setup --content-repository default -> error
2. flow cr:replayall --content-repository default -> error
3. flow cr:setup --content-repository default -> okay
4. flow cr:replayall --content-repository default -> okay

if we are okay with this, as there are way more breaking changes already ahead - like the workspace thing - which absolutely crushes an installation we can just move forward and i +1.

[kitsunet]

I guess one just tries and fails to update the schema, the replay clears the tables and errors out on the first write due to wrong schema, then setup works because no data, and then replay works.

[mhsdesign]

hum im stupid this sound 100% logical. Thanks for letting me use your brain 😂

[mhsdesign]

but doesnt this scream for a cr:setup --empty-tables-except-the-events or cr:prune --keep-event-log or the like?

[kitsunet]

would be neat for sure, but IDK if we shouldn't keep the schema stable once we have 9.0 out the door

[kitsunet]

would be need for sure, but IDK if we shouldn't keep the schema stable once we have 9.0 out the door

[mhsdesign]

Well this is comforting. It actually makes the db even easier to debug by hand :D

before in cr_default_p_graph_node

before in cr_default_p_graph_hierarchyrelation

---

after in cr_default_p_graph_node

after in cr_default_p_graph_hierarchyrelation

[mhsdesign]

Updated the upgrade instructions and made this followup for a cr:resetall #4864 (please review :D)
Either way this looks neat and works so lets go