Skip to content

orm: null handling and optional fields - #19379

Merged
spytheman merged 39 commits into
vlang:masterfrom
edam:pr-orm-null-handling
Oct 5, 2023
Merged

spytheman merged 39 commits into
vlang:masterfrom
edam:pr-orm-null-handling

Conversation

@edam

@edam edam commented Sep 18, 2023 •

Copy link
Copy Markdown
Member

This PR adds:

  • Option fields in ORM table structs
    • CREATE TABLE fields NOT NULL and unless they are option fields
    • ORM layer updated to handle wrapping/unwrapping option fields
  • NULL handling
    • Adds an orm.Null type
    • DB drivers updated to receive/emit NULLs
    • handle is none and !is none in WHERE clauses
    • handle UPDATE expressions with none (e.g., update Foo, set a = none where...)
  • Fixes (not) setting of fields during INSERT
    • New logic is: default values in serial fields and fields with default attr are not INSERTed

Apologies for lumping so much stuff together! They are related changes. (I.e., I couldn't implement NULL handling without option fields.)

🤖 Generated by Copilot at 377b1a3

This pull request enhances the ORM module by generating more readable and consistent C code, and by supporting more features and types for database operations. It also refactors some of the ORM code generation logic for clarity and simplicity.

🤖 Generated by Copilot at 377b1a3

  • Simplify and improve the generated C code for ORM queries by adding indentation, newlines, and error handling, and avoiding unnecessary string concatenation and unwrapping (link, link, link, link, link, link, link, link, link, link)
  • Support the new ORM features for checking null values in the database by adding cases for is and not_is operations, and handling option types and time.Time type in the data array and the result variable (link, link, link, link)
  • Support the new ORM features for handling auto-increment and default fields in the insert and update queries by finding the indexes of those fields and skipping them in the query parameters (link)
  • Optimize and simplify the logic for finding the table, sql, and primary attributes in the struct info and the fields by using the find_first, contains, and contains_arg methods instead of looping over the attributes manually (link, link)

edam added 21 commits September 3, 2023 09:30
Nullable DB fields are now determined by corresponding option struct field.  The
[nonull] attribute is deprecated and fields are all NOT NULL now, unless they
are option fields. New orm primitive, NullType, added to support passing none
values to db backends, which have been updated to support it.  Also, empty
string and 0 numberic values are no longer skipped during insert (since they may
be valid values).
also, improved formatting for orm cgen, and removed optimised operand handling
of orm `is` and `!is` operators
During insert, fields which are
* [sql: serial]
* [default: whatever]
and where the data is a default value (e.g., 0, ""), those fields are not sent
to the db, so that the db can generate auto-increment or default values.  (This
was previously done only for [primary] fields, and not in all circumstances, but
that is not correct -- primary and serial/auto-increment fields are differnet.)
Changed orm.write_orm_select() so that you pass to it the name of a resut
variable which it populates with the result (or not) and changed use of it in
sql_select_expr() and calls in write_orm_select() to populate substructs.
@edam

edam commented Sep 18, 2023

Copy link
Copy Markdown
Member Author

I am working through resolving conflicts with changes made to master... 😢

@medvednikov

Copy link
Copy Markdown
Member

This is great stuff!

Thanks, Tim.

Comment thread vlib/orm/orm.v Outdated
@spytheman
spytheman marked this pull request as draft September 19, 2023 11:03
@edam
edam force-pushed the pr-orm-null-handling branch from 311bb0b to 3532ab7 Compare September 21, 2023 20:36
Code in checker/orm.v used the SqlStmtLine object field name to store c-specific
referenecs to option and array fields (for arrays of children).  I moved this
logic to cgen.  And fixed an issue introduced with option fields, where an array
of children was unpacked into a non-array result which could corrupt memory.
@edam
edam force-pushed the pr-orm-null-handling branch from 50d5e43 to c1df21a Compare October 3, 2023 21:21
@edam edam changed the title [WIP] orm: null handling and optional fields orm: null handling and optional fields Oct 3, 2023
@edam
edam marked this pull request as ready for review October 5, 2023 11:07

@spytheman spytheman left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent work.

@spytheman
spytheman force-pushed the pr-orm-null-handling branch from a306df7 to 8b027b4 Compare October 5, 2023 15:49
…tes.vv; run `VAUTOFIX=1 ./v vlib/v/compiler_errors_test.v`
@spytheman
spytheman merged commit 7560753 into vlang:master Oct 5, 2023
Wertzui123 pushed a commit to Wertzui123/v that referenced this pull request Oct 8, 2023
* orm: added is none and !is none handling

* orm: added NullType, support option fields and deprecate [nonull]

Nullable DB fields are now determined by corresponding option struct field.  The
[nonull] attribute is deprecated and fields are all NOT NULL now, unless they
are option fields. New orm primitive, NullType, added to support passing none
values to db backends, which have been updated to support it.  Also, empty
string and 0 numberic values are no longer skipped during insert (since they may
be valid values).

* orm: fix [nonull] deprecation warning

* orm: add null handling to update and select

also, improved formatting for orm cgen, and removed optimised operand handling
of orm `is` and `!is` operators

* sqlite: read/report NULLs using new orm NullType

* postgres: returning data primitives now returns new orm.NullType

* orm: initialise NullType Primitives properly

* orm: do not smart cast operands inside sql

* orm: fix bad setting of option value

* orm: improve orm_null_test.v, adding/fixing selects

* orm: cleanup: rename NullType->Null, use serial const, cgen output

* orm: handle automatically generated fields more explicitly

During insert, fields which are
* [sql: serial]
* [default: whatever]
and where the data is a default value (e.g., 0, ""), those fields are not sent
to the db, so that the db can generate auto-increment or default values.  (This
was previously done only for [primary] fields, and not in all circumstances, but
that is not correct -- primary and serial/auto-increment fields are differnet.)

* orm: udpated README

* orm: select cgen fixes: read from uninit res; fail to init res

* orm: udpated tests

* orm: fix option sub-struct fields

* orm: fixed joins to option structs

Changed orm.write_orm_select() so that you pass to it the name of a resut
variable which it populates with the result (or not) and changed use of it in
sql_select_expr() and calls in write_orm_select() to populate substructs.

* orm: fix pg driver handling of NULL results

* orm: move runtime checks to comptime checker; cache checked tables

* orm: vfmt :(

* orm: markdown formatting

* orm: renamed orm.time_ and orm.enum_; updated db drivers

* checker: updated orm tests

* orm: fix issue setting up ast option values as orm primitives

* checker: ORM use of none/options and operations (added tests)

* orm: fixed tests

* db: clean code

* examples: remove orm nonull attributes

* orm: skip test memory santisation for orm_null_test.v

* orm: make the type-to-primitive converstion fns not public

* orm: mv object var c-code from checker->cgen; fix memory corruption

Code in checker/orm.v used the SqlStmtLine object field name to store c-specific
referenecs to option and array fields (for arrays of children).  I moved this
logic to cgen.  And fixed an issue introduced with option fields, where an array
of children was unpacked into a non-array result which could corrupt memory.

* orm: fixed vast error

* orm: skip 2 tests on ubuntu-musl which require sqlite3.h

* cgen: prevent casting a struct (string)

* v fmt orm_fkey_attribute.vv, orm_multidim_array.vv, orm_table_attributes.vv; run `VAUTOFIX=1 ./v vlib/v/compiler_errors_test.v`
juan-db added a commit to juan-db/v that referenced this pull request Dec 28, 2023
The @[nonull] attribute is no longer used and is replaced by using an option type.
vlang#19379
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants