🔥 Jadwal & Hasil: vlang/v/pull/

Casper64 · 2023-06-20T23:30:35Z

Reimplement the picoev and picohttpparser module in V.

This pullrequest aims to reimplement the picoev and picohttpparser module in V. If it's fully written in V it is easier to contribute for others to improve the module. It will also get rid of a third party depency.

Currently the module works on Linux with epoll and on windows with select.
The select version works on all platforms that support select but for now it's exclusive to windows for testing purposes.

Usage

module main

import picoev
import picohttpparser
import net.http

fn main() {
	mut server := picoev.new(picoev.Config{
		cb: handle_request
	})
	server.serve()
}

fn handle_request(userdata voidptr, req picohttpparser.Request, mut res picohttpparser.Response) {
	res.http_ok()
	res.html()
	res.body('Hello Picoev!')
	res.end()
}

Performance

Performance is as fast, if not faster, as with the C implementation

TODO:

Implemt epoll for (linux)
Implement select for windows
Implement kqueue (MacOS & FreeBSD)
Make select also available for unix systems.
Make kqueue available on all operating systems that support it.
Reimplement picohttparser.Request in V

And other optimizations.

Contribution needed

I am working on the kqueue implementation, but since I don't own a mac I can't test it on macos 🥲

`🤖 Generated by Copilot at a19651c`

This pull request implements a custom Picoev struct and event loop for the picoev library, and improves the Windows compatibility of the picohttpparser library. It refactors the picoev.v file and adds the multiplex_linux.v and multiplex_windows.v files for different event loop implementations. It also changes the response.v file to use the C.send function for sending data.

`🤖 Generated by Copilot at a19651c`

Add epoll-based event loop for Linux systems (link)
Add select-based event loop for Windows and other systems (link)
Refactor picoev.v file to use Picoev struct and methods instead of C.picoev functions and types (link,link,link,link,link,link,link,link,link,link)
Update response.v file to use C.send function instead of C.write function for Windows compatibility (link)

JalonSolov · 2023-06-20T23:53:28Z

Can you also refactor the C. stuff into .c.v files?

spytheman · 2023-06-27T09:13:14Z

 fn json_response() string {
 	msg := Message{
-		message: 'Hello, World!'
+		message: 'Hello, World!\n'


Why add the \n? Imho it is not related to the rewrite, and will make performance comparisons with the older C implementation slightly harder (the size of the response will increase).

spytheman · 2023-06-27T09:15:12Z


+// ev_set sets a new `kevent` with file descriptor `index`
 [inline]
-pub fn (mut pv Picoev) ev_set(index int, operation int, events int) {


Would not that method be useful as a pub in the future as well?

You're right! Don't know why I removed that

JalonSolov · 2023-06-27T17:17:32Z

Any benchmarks available? It would be interesting to see, even if only for intermediate changes.

Of course the code can always be optimized later (and the benchmarks could be run twice... once normally, and once with -prod).

Casper64 · 2023-06-27T21:30:22Z

@JalonSolov I've done quite some testing for fun. Below are some benchmarks using wrk and baton (a go testing tool).

For each benchmark I took the best out of 3.
Everything is tested on linux WSL2 since the old Picoev didn't work with Windows and I don't own a mac :).
CPU: Intel Core i9 12900K (24 threads).

Summary

V version is slightly faster with -prod, but slower without -prod.

Memory usage is the 1-4mb higher on the new version.

Sending connections from multiple threads.

I found that wrk results differ too much on each run when sending requests from multiple threads, that is why I used baton to benchmark sending connections from multiple threads.

Command used

sending requests from 100 concurrent connections for 30 seconds, with goroutines so covering multiple threads.

baton -z requests.csv -c 100 -t 30

requests.csv

GET,http://127.0.0.1:8088/,,,
GET,http://127.0.0.1:8088/t,,,

Old

With prod:

└─$ baton -z requests.csv -c 100 -t 30
Configuring to send requests from file. (Read 2 requests)
Generating the requests...
Finished generating the requests
Sending the requests to the server...
Finished sending the requests
Processing the results...


====================== Results ======================
Total requests:                              10413565
Time taken to complete requests:        30.003644803s
Requests per second:                           347077
===================== Breakdown =====================
Number of connection errors:                        0
Number of 1xx responses:                            0
Number of 2xx responses:                     10413565
Number of 3xx responses:                            0
Number of 4xx responses:                            0
Number of 5xx responses:                            0
=====================================================

Without prod:

└─$ baton -z requests.csv -c 100 -t 30
Configuring to send requests from file. (Read 2 requests)
Generating the requests...
Finished generating the requests
Sending the requests to the server...
Finished sending the requests
Processing the results...


====================== Results ======================
Total requests:                               9331613
Time taken to complete requests:        30.003814517s
Requests per second:                           311014
===================== Breakdown =====================
Number of connection errors:                        0
Number of 1xx responses:                            0
Number of 2xx responses:                      9331613
Number of 3xx responses:                            0
Number of 4xx responses:                            0
Number of 5xx responses:                            0
=====================================================

New

With prod:

└─$ baton -z requests.csv -c 100 -t 30
Configuring to send requests from file. (Read 2 requests)
Generating the requests...
Finished generating the requests
Sending the requests to the server...
Finished sending the requests
Processing the results...


====================== Results ======================
Total requests:                              10713605
Time taken to complete requests:        30.003676865s
Requests per second:                           357076
===================== Breakdown =====================
Number of connection errors:                        0
Number of 1xx responses:                            0
Number of 2xx responses:                     10713605
Number of 3xx responses:                            0
Number of 4xx responses:                            0
Number of 5xx responses:                            0
=====================================================

Without prod:

└─$ baton -z requests.csv -c 100 -t 30
Configuring to send requests from file. (Read 2 requests)
Generating the requests...
Finished generating the requests
Sending the requests to the server...
Finished sending the requests
Processing the results...


====================== Results ======================
Total requests:                               7962170
Time taken to complete requests:          30.0039103s
Requests per second:                           265371
===================== Breakdown =====================
Number of connection errors:                        0
Number of 1xx responses:                            0
Number of 2xx responses:                      7962170
Number of 3xx responses:                            0
Number of 4xx responses:                            0
Number of 5xx responses:                            0
=====================================================

Sending connections from a single thread

Since Picoev is a single threaded application it is fair to also test the performance when connections come from a single thread.

Command used

sending requests from 1000 concurrent connections for 30 seconds from a single thread.

wrk -t 1 -c 1000 http://127.0.0.1:8088/ -d 30s

Old

With prod:

─$ wrk -t 1 -c 1000 http://127.0.0.1:8088/ -d 30s
Running 30s test @ http://127.0.0.1:8088/
  1 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.52ms  393.04us   6.40ms   87.68%
    Req/Sec     1.05M    29.71k    1.11M    87.96%
  31201845 requests in 30.08s, 2.27GB read
Requests/sec: 1037353.59
Transfer/sec:     77.17MB

New

With prod:

└─$ wrk -t 1 -c 1000 http://127.0.0.1:8088/ -d 30s
Running 30s test @ http://127.0.0.1:8088/
  1 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.48ms  329.80us   6.83ms   90.04%
    Req/Sec     1.05M    33.61k    1.11M    85.95%
  31447155 requests in 30.08s, 2.28GB read
Requests/sec: 1045538.12
Transfer/sec:     77.77MB

Windows

For the windows fans: select could only handle 20 concurrent connections much more and the program segfaults. I tried increasing the maximum open file descriptors and turning of defender, but it seems this is "just windows".

>baton.exe -z requests.csv -c 20 -t 30
Configuring to send requests from file. (Read 2 requests)
Generating the requests...
Finished generating the requests
Sending the requests to the server...
Finished sending the requests
Processing the results...


====================== Results ======================
Total requests:                               3333825
Time taken to complete requests:          30.0013945s
Requests per second:                           111122
===================== Breakdown =====================
Number of connection errors:                        0
Number of 1xx responses:                            0
Number of 2xx responses:                      3333825
Number of 3xx responses:                            0
Number of 4xx responses:                            0
Number of 5xx responses:                            0
=====================================================

Multithreaded epoll

Since epoll and kqueue can be accessed from multiple threads it is easy to add it in Picoev, I made a simple implementation.

The program runs on 10 threads: 1 for accepting connections, 8 workers for processing the handlers, 1 for passing the accepted connections to the 8 workers.

└─$ wrk -t 12 -c 1000 http://127.0.0.1:8088/ -d 30s
Running 30s test @ http://127.0.0.1:8088/
  12 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   429.41us  199.05us  11.48ms   74.64%
    Req/Sec   465.03k    23.37k  673.33k    72.40%
  166797606 requests in 30.10s, 12.12GB read
Requests/sec: 5541659.51
Transfer/sec:    412.23MB

JalonSolov · 2023-06-27T22:39:01Z

Very nice numbers!

S-YOU

Awesome work!

S-YOU · 2023-07-01T08:48:52Z

I found that wrk results differ too much on each run when sending requests from multiple threads, that is why I used baton to benchmark sending connections from multiple threads.

I think if you use taskset on linux for multithreaded version, you could probably get better results, like 10 cpus (also 10 threads) to server and 14 cpus to wrk, for example (for 24 cpus machine).

taskset -c 14-23 ./server ...
taskset -c 0-13 wrk ...

JalonSolov · 2023-07-08T01:18:24Z


 	pret.ret = -1
-	return
+	return error('error parsing request: no request foud')


Suggested change

return error('error parsing request: no request foud')

return error('error parsing request: no request found')

JalonSolov · 2023-07-08T01:19:55Z

 	} else {
 		pret.ret = -1
-		return unsafe { nil }
+		return error('error parsing request: no newline after http version')


Perhaps EOL instead of newline, since it should always be carriage return + newline (\r\n)?

Or just "\r\n" as you use in a later message.

spytheman · 2023-07-08T04:23:02Z

@Casper64 is there a performance impact from the last commit fcb5673 ?

I am mostly curious, because of this:

- fn (mut r Request) phr_parse_request_path(buf_start &u8, buf_end &u8, mut pret Pret) {
+ fn (mut r Request) phr_parse_request_path(buf_start &u8, buf_end &u8, mut pret Pret) ! {

Casper64 · 2023-07-08T10:20:59Z

@spytheman interestingly the performance difference is very minimal (1-2%) when the request parsing is done inside the picoev loop, which I tested yesterday.

However I just did some benchmarking with plain parsing the request and that’s almost 2 times slower 🤯, so I will revert the change.

spytheman · 2023-07-11T16:16:22Z

@Casper64 can it be merged as it is?

Casper64 · 2023-07-11T18:37:18Z

@Casper64 can it be merged as it is?

Should the thirdparty dependency be removed in this pr? If no then it can be merged.

JalonSolov · 2023-07-11T18:57:37Z

Baby steps. I'd say merge this, and do another PR to clean it up, optimize it, or whatever. Can also wait for the Mac kqueue support, etc.

spytheman · 2023-07-12T06:39:45Z

I am merging it, since it is already quite useful on Linux as it is.

Casper64 · 2023-07-12T13:22:49Z

@JalonSolov I'll update the pr description, but mac is supported as is freebsd with kqueue. Any other OS apart from linux will use select.

Casper64 added 2 commits June 21, 2023 01:05

upload linux and windows code

0309a5b

update comments

a19651c

Casper64 added 27 commits June 22, 2023 13:42

first stage kqueue

af7137b

add different stuct name

053c972

fix format

90561c8

bug1

4d67cf7

bug2

e880bb4

bug3

d4368a8

bug4

5f83f4e

bug6

d42fad0

bug7

10cf958

bug8

6fa436f

bug9

438a37d

bug10

fc7bbaa

bug11

defd35b

bug12

403ce36

bug13

dcc93aa

bug14

5fb1a34

bug15

ed38ff5

bug17

ce24dd1

bugs17

8a8ad34

bug18

9aeefce

bug18

a39a020

fix

971eadc

more bugs

d171d5e

fix

30d99ad

use macro

a3fc185

Merge branch 'master' into picoev_mac

61f5576

rebase

fb885ca

Casper64 added 6 commits June 26, 2023 13:19

use default C names

29943d1

bug37

8d4d630

cleanup

109dd3f

name changes

bf2ed94

fix

84a9607

added kqueue

964941c

spytheman reviewed Jun 27, 2023

View reviewed changes

S-YOU approved these changes Jul 1, 2023

View reviewed changes

Casper64 added 3 commits July 2, 2023 18:10

apply reviews, make loop and event structs public

efccc77

rewrite picohttpparser Request in V and remove C code

62d4daf

add errors for picohttpparser + fixes

fcb5673

Casper64 marked this pull request as ready for review July 7, 2023 21:30

JalonSolov reviewed Jul 8, 2023

View reviewed changes

Casper64 added 2 commits July 8, 2023 20:20

picohttpparser: only use error at Request & apply reviews

433861f

revert speed regressions

83ed085

spytheman merged commit a43064a into vlang:master Jul 12, 2023

Casper64 deleted the picoev branch November 20, 2023 23:12

	return error('error parsing request: no request foud')
	return error('error parsing request: no request found')

Conversation

Casper64 commented Jun 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Usage

Performance

TODO:

Contribution needed

🤖 Generated by Copilot at a19651c

🤖 Generated by Copilot at a19651c

Uh oh!

JalonSolov commented Jun 20, 2023

Uh oh!

spytheman Jun 27, 2023

Choose a reason for hiding this comment

Uh oh!

spytheman Jun 27, 2023

Choose a reason for hiding this comment

Uh oh!

Casper64 Jun 27, 2023

Choose a reason for hiding this comment

Uh oh!

JalonSolov commented Jun 27, 2023

Uh oh!

Casper64 commented Jun 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Sending connections from multiple threads.

Command used

Old

New

Sending connections from a single thread

Command used

Old

New

Windows

Multithreaded epoll

Uh oh!

JalonSolov commented Jun 27, 2023

Uh oh!

S-YOU left a comment

Choose a reason for hiding this comment

Uh oh!

S-YOU commented Jul 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JalonSolov Jul 8, 2023

Choose a reason for hiding this comment

Uh oh!

JalonSolov Jul 8, 2023

Choose a reason for hiding this comment

Uh oh!

JalonSolov Jul 8, 2023

Choose a reason for hiding this comment

Uh oh!

spytheman commented Jul 8, 2023

Uh oh!

Casper64 commented Jul 8, 2023

Uh oh!

spytheman commented Jul 11, 2023

Uh oh!

Casper64 commented Jul 11, 2023

Uh oh!

JalonSolov commented Jul 11, 2023

Uh oh!

spytheman commented Jul 12, 2023

Uh oh!

Casper64 commented Jul 12, 2023

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Casper64 commented Jun 20, 2023 •

edited

Loading

`🤖 Generated by Copilot at a19651c`

`🤖 Generated by Copilot at a19651c`

Casper64 commented Jun 27, 2023 •

edited

Loading

S-YOU commented Jul 1, 2023 •

edited

Loading