Skip to content

json2: strict module - #17927

Merged
spytheman merged 19 commits into
vlang:masterfrom
enghitalo:json2/strict_decode
Jan 1, 2024
Merged

spytheman merged 19 commits into
vlang:masterfrom
enghitalo:json2/strict_decode

Conversation

@enghitalo

@enghitalo enghitalo commented Apr 10, 2023 •

Copy link
Copy Markdown
Contributor

This PR

fn test_strict_check() {
	assert strict.strict_check[StructTypeAndOptionType[string]]('{"val": "","val": ""}') == strict.StructCheckResult{
		duplicates: ['val']
		superfluous: []
	}

	assert strict.strict_check[StructTypeAndOptionType[string]]('{"val": "","val2": ""}') == strict.StructCheckResult{
		duplicates: []
		superfluous: ['val2']
	}
}

Next PRs

strict_decode

  • Stop on first error to be performative
struct StructType {
mut:
	val        string
}

fn test_superfluous() {
	json.strict_decode[StructType]('{"val": "","val2": ""}') or {
		assert err.msg() == '[x.json2] useless data detected. Try pruning superfluous fields shown in err.superfluous_fields()'
		assert err.superfluous_fields() == ['val2']
		return
	}
	assert false
}

prune

struct StructType {
mut:
	val        string
}

fn test_superfluous() {
	json.prune[StructType]('{"val": "","val2": ""}') == '{"val": ""}'
}

Performance improvments

@enghitalo
enghitalo marked this pull request as draft April 10, 2023 13:24
Comment thread vlib/x/json2/decode_strict_struct_test.v Outdated
Co-authored-by: JalonSolov <JalonSolov@gmail.com>
Comment thread vlib/x/json2/decode_strict_struct_test.v Outdated
Comment thread vlib/x/json2/decode_strict_struct_test.v Outdated
enghitalo and others added 3 commits April 19, 2023 06:38
Co-authored-by: Turiiya <34311583+tobealive@users.noreply.github.com>
Co-authored-by: Turiiya <34311583+tobealive@users.noreply.github.com>
@spytheman spytheman added Needs Rebase The code of the PR must be rebased over current master before it can be approved. Breaking Change This PR introduces changes that break backward compatibility. Requires manual review. labels Aug 21, 2023
@enghitalo
enghitalo marked this pull request as ready for review December 10, 2023 17:53
@enghitalo
enghitalo requested a review from ttytm December 10, 2023 17:56
@enghitalo enghitalo changed the title json2: strict_decode json2: strict module Dec 10, 2023
@enghitalo

Copy link
Copy Markdown
Contributor Author

How can this CI fail can be related to this PR?
image

Comment thread vlib/x/json2/strict/strict.v Outdated
Comment thread vlib/x/json2/strict/strict.v Outdated
Comment thread vlib/x/json2/strict/strict.v Outdated
Comment thread vlib/x/json2/strict/strict.v Outdated
Comment on lines +23 to +31
// REVIEW how performatic is it?
fn strict_check[T](json_data string) StructCheckResult {
$if T is $struct {
tokens := tokenize(json_data)

key_struct := get_keys_from_json(tokens)

mut duplicates := get_duplicates_keys(tokens, key_struct)
mut superfluous := get_superfluous_keys[T](tokens, key_struct)

@ttytm ttytm Dec 11, 2023 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing that comes to my mind would be to get the duplicates and superfluous in one loop. Both are recursively called, so it could be worth a try using one instead of multiple consecutive loops. Did you try that?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will try it in next PR. Thank you!!

@ttytm

ttytm commented Dec 11, 2023 •

Copy link
Copy Markdown
Member

How can this CI fail can be related to this PR?

Good work! It's likely not related πŸ™‚ πŸ‘

@enghitalo

Copy link
Copy Markdown
Contributor Author

Thank you for review @ttytm !

@spytheman

Copy link
Copy Markdown
Contributor

I do not understand, what is the use case of this submodule?

Comment thread vlib/x/json2/strict/strict.v Outdated
Comment thread vlib/x/json2/strict/strict.v Outdated
@spytheman

Copy link
Copy Markdown
Contributor

I think we have a bug in the compiler, that allowed non public functions to be called by the test, without it being marked as an internal one.

@enghitalo

enghitalo commented Dec 17, 2023 •

Copy link
Copy Markdown
Contributor Author

I do not understand, what is the use case of this submodule?

Many of the companies I worked for had slowdown issues due to excessive unnecessary data sent through the back-end of web applications. Using a library like this, on the front-end, for example, helps keep the back-end code fast, efficient, safe and with better-defined routes. Which results in a lower cost for projects in production

@spytheman

Copy link
Copy Markdown
Contributor

I still do not understand the usecase πŸ€” .

Lets take for example a web app, and let it have:
a) a JS client running in a browser,
b) a backend webserver implemented in V, connected to
c) a DB, say MySQL.

The communication between the JS client and the backend in V, happens by sending JSON messages.

Where would such a module be used?
aa) when the V webserver sends JSON to the JS client?
bb) when the V webserver receives JSON from the JS client?

If aa) is the case, could not the webserver just not generate the additional keys, in the first place, and that would have been caught in code review?

If bb) is the case, what would the webserver do, when it detects it? Log that it found duplicated keys? Who will read those logs? Or will it stop processing, until the JS client is fixed?

@spytheman

Copy link
Copy Markdown
Contributor

I also could not understand how is that related to routing at all πŸ€” .

enghitalo and others added 2 commits December 18, 2023 10:43
Co-authored-by: Delyan Angelov <delian66@gmail.com>
Co-authored-by: Delyan Angelov <delian66@gmail.com>
@enghitalo
enghitalo marked this pull request as draft December 20, 2023 14:06
@enghitalo

Copy link
Copy Markdown
Contributor Author

Hi. Sorry the delay. I will answer everything As Soon As Possible

@enghitalo

Copy link
Copy Markdown
Contributor Author

I also could not understand how is that related to routing at all πŸ€” .

It is very common for some developers to use a single route for multiple purposes, what is no good. For example, a route that returns all the products in the database, sometimes even with relationships, when, in many cases, they do not need all this information. Use the strict library to check that there is no unnecessary data, to perform end-to-end tests, send a log, or even return an error. It would help to create and maintain a final product with greater quality and performance, create a quality metric for the development team, facilitate code review...

@enghitalo
enghitalo marked this pull request as ready for review December 29, 2023 13:39
@enghitalo

Copy link
Copy Markdown
Contributor Author

I still do not understand the usecase πŸ€” ....

More the bb) case.

It will depend on how rigid the team is. Personally, I would use Log Management or apply it to tests. Since, for me, it wouldn't be very interesting to break the application with errors that could be random (But it also depends on how these errors will be handled)

@spytheman
spytheman merged commit cb28144 into vlang:master Jan 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Breaking Change This PR introduces changes that break backward compatibility. Requires manual review. Needs Rebase The code of the PR must be rebased over current master before it can be approved.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants