<crystalmq>
@bararchy sleep 1 seems to yield the same results.... but the effect doesn't change.
<FromGitter>
<bararchy> crystalmq: Does your issue is that it's blocking ?
<crystalmq>
bararchy: It seems like it, I would expect the requests to be stable - but it seems like if I have two sockets inputting and outputting at a high volume, only one socket gets processor time, the other doesn't
<crystalmq>
I'm not exactly sure what is causing it, I'm quite new to Crystal, but I am a rubyist.
<crystalmq>
The reason I noticed this, is because I was benchmarking a test message queue written in crystal and noticed when the requests got high enough this effect happened, eventually it would get killed by the OOM killer.
<crystalmq>
So, i tried to reproduce the problem in the simplest form... which i have done thankfully.
<crystalmq>
But this is way beyond my knowledge level, so i thought to reach out to the community before creating a git issue to make sure it wasnt my fault.
<FromGitter>
<bararchy> crystalmq: I commented on both your codes
<FromGitter>
<bararchy> in the gist
<FromGitter>
<bararchy> try those samples and tell me if you see any change
<crystalmq>
@bararchy testing now
<FromGitter>
<bararchy> Now, usually sockets in crystal (as oppose to ruby defaults) are non_blocking, so a socket.read will return with or without any data, just poitning it out
<FromGitter>
<bararchy> this is whay a good thing is to check if `socket.read > 0`
<crystalmq>
"Your account has been flagged. Because of that, your profile is hidden from the public. If you believe this is a mistake, contact support to have your account status reviewed." :(
<FromGitter>
<bararchy> Oo
<FromGitter>
<bararchy> wierd
<crystalmq>
Very weird.
<FromGitter>
<bararchy> anyway, why is the sleep there ?
<crystalmq>
The sleep is there because without it - you get the second example
<crystalmq>
Where, there's a large number of requests then... 0
<FromGitter>
<bararchy> hm.....
<crystalmq>
It seems to give the inbound socket time to 'release' to allow other things to be processed
<FromGitter>
<bararchy> Oh
<FromGitter>
<bararchy> I know
<crystalmq>
if i dont add the sleep, it will never stop processing the inbound data
soveran has quit [Remote host closed the connection]
leafybasi has quit [Ping timeout: 260 seconds]
mooe has quit [Quit: Connection closed for inactivity]
gloscombe has quit [Ping timeout: 258 seconds]
leafybasi has joined #crystal-lang
gloscombe has joined #crystal-lang
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
snsei has joined #crystal-lang
pawnbox has quit [Remote host closed the connection]
snsei has quit [Ping timeout: 245 seconds]
crystalmq has quit [Ping timeout: 260 seconds]
p0p0pr37_ has joined #crystal-lang
p0p0pr37 has quit [Read error: Connection reset by peer]
p0p0pr37_ is now known as p0p0pr37
bjz has quit [Ping timeout: 256 seconds]
bjz has joined #crystal-lang
soveran has joined #crystal-lang
soveran has joined #crystal-lang
soveran has quit [Changing host]
soveran has quit [Remote host closed the connection]
soveran has joined #crystal-lang
soveran has joined #crystal-lang
soveran has quit [Changing host]
snsei has joined #crystal-lang
snsei has quit [Ping timeout: 240 seconds]
<crystal-gh>
[crystal] rbviz opened pull request #3678: Fix SDL samples on linux (master...fix-sdl-samples-on-linux) https://git.io/v1K5e
crystalmq has joined #crystal-lang
<crystalmq>
@RX14 @unshadow
<crystalmq>
Problem solved.
<crystalmq>
It was the event loop being exhausted. I had an erroneous spawn inside the message distribution block that was starting a new fiber for each message being sent.
<crystalmq>
When we reached 150k/messages a second, there wasn't enough room left to process anything else, so it ground to a halt. After removing that event block we're happing pushing 300k/messages a second now.
<crystalmq>
*spawn
z64 has joined #crystal-lang
gloscombe has quit [Quit: Lost terminal]
pawnbox has joined #crystal-lang
gloscombe has joined #crystal-lang
<Yxhuvud>
Suppose I have an ordered array. What is the most efficient way to insert a new number into that? (or alternatively, so we have any better data structure for that?)
<Yxhuvud>
or even more alternatively speaking, if we have a priority queue then that would be even neater
<BlaXpirit>
Yxhuvud, if the insertions tend towards the beginning or the end, Deque can outperform Array. but if it's uniform, then it probably won't
<Yxhuvud>
good to know that external resources like that exist, even if I need to keep to stdlib in this particular case
mgarciaisaia has joined #crystal-lang
<FromGitter>
<spalladino> @crystalmq those are truly awesome numbers! Are you planning to run it on a production environment? If so, do send us a shout-out, we'd love to add it to the list.
gloscombe has quit [Ping timeout: 250 seconds]
gloscombe has joined #crystal-lang
snsei has joined #crystal-lang
snsei has quit [Ping timeout: 245 seconds]
mgarciaisaia has quit [Ping timeout: 260 seconds]
pduncan has joined #crystal-lang
crystalmq has quit [Ping timeout: 260 seconds]
vivus-ignis has joined #crystal-lang
mark_66 has quit [Remote host closed the connection]
mgarciaisaia has joined #crystal-lang
mark_66 has joined #crystal-lang
<FromGitter>
<bararchy> > **<crystalmq>** When we reached 150k/messages a second, there wasn't enough room left to process anything else, so it ground to a halt. After removing that event block we're happing pushing 300k/messages a second now. ⏎ ⏎ ⏎ How did you remove the event block ? [https://gitter.im/crystal-lang/crystal?at=584ec738be9d43bc634ec0f3]
dostoyevsky has quit [Quit: leaving]
dostoyevsky has joined #crystal-lang
asterite has quit [Ping timeout: 250 seconds]
DeBot has quit [Ping timeout: 260 seconds]
jhass has quit [Ping timeout: 268 seconds]
jhass has joined #crystal-lang
asterite has joined #crystal-lang
leafybasi has quit [Ping timeout: 258 seconds]
pawnbox has quit [Read error: Connection reset by peer]
pawnbox has joined #crystal-lang
leafybasi has joined #crystal-lang
<FromGitter>
<samueleaton> Is there a Crystal parser comparable to https://github.com/ternjs/acorn that breaks files down into tokens and whatnot? The goal is to build some more tooling on top of Crystal.
<RX14>
@samueleaton you can use the crystal lexer
<RX14>
just require the parts you need from the crystal compiler
<FromGitter>
<samueleaton> @spalladino I was the last person to commit to that file and didn't even remember until I saw some code that looked too familiar haha! 😆 Thanks! very helpful.
<FromGitter>
<spalladino> Hahahahah I didn't notice your name at the top :-P
lacour has joined #crystal-lang
<crystal-gh>
[crystal] Sija opened pull request #3680: Implement HTTP::WebSocket#on_{ping,pong} handler (master...websocket-ping) https://git.io/v16Mr
<crystal-gh>
[crystal] trans closed pull request #3608: YAML support for emitting tags and style (master...emit-tags) https://git.io/v1Y1v
gloscombe has quit [Quit: Lost terminal]
sp4rrow has joined #crystal-lang
mark_66 has quit [Remote host closed the connection]
Ven has joined #crystal-lang
pawnbox_ has joined #crystal-lang
pawnbox has quit [Read error: No route to host]
Ven has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<crystal-gh>
[crystal] asterite pushed 4 new commits to master: https://git.io/v16hj
<crystal-gh>
crystal/master 08fa553 Ary Borenszweig: Don't consider var as nilable if conditionally assigned inside initialize, but has initializer. Fixes #3669
<crystal-gh>
[crystal] asterite pushed 1 new commit to master: https://git.io/v16jU
<crystal-gh>
crystal/master a7135c8 TSUYUSATO Kitsune: Allow to pass IO to from_yaml
pawnbox_ has quit [Remote host closed the connection]
<crystal-gh>
[crystal] asterite pushed 1 new commit to master: https://git.io/v16js
<crystal-gh>
crystal/master dfc4367 TSUYUSATO Kitsune: Clean up empty line on crystal init lib
<crystal-gh>
[crystal] Sija opened pull request #3681: Use Slice.empty and Bytes consistently throughout the codebase (master...slice-empty) https://git.io/v1ieR
sp4rrow has quit [Read error: Connection reset by peer]
sp4rrow has joined #crystal-lang
sp4rrow has quit [Quit: The Internet needs a break and I need a cookie]
<crystal-gh>
[crystal] asterite pushed 1 new commit to master: https://git.io/v1iBz
<crystal-gh>
crystal/master f850fa0 Ary Borenszweig: Travis: don't run in verbose mode anymore
<FromGitter>
<taylorfinnell> hello! is the verbose fomatter (crystal spec -v) excluded from the help list (crystal spec -h) for some reason? it looks to me like it may have some formatting issues so perhaps that is why it is not documented
<RX14>
you need to use crystal spec -- --help
<RX14>
there are two sets of options
<RX14>
options for the compiler, and options for the spec executable
<FromGitter>
<taylorfinnell> sorry, I was not clear. I am wondering why the verbose spec formatter is not documented. why is it not in the available options for crystal spec -h
<RX14>
because those options are the compiler help
<RX14>
and only shows the compiler options
<RX14>
verbose is a spec option
<FromGitter>
<taylorfinnell> ohh
<FromGitter>
<taylorfinnell> i totally missed your `crystal spec -- --help` comment
<FromGitter>
<taylorfinnell> so sorry! thank you!
<crystal-gh>
[crystal] asterite closed pull request #3621: Add missing --no-color option to spec and eval command (master...fix/add-no-color-option-to-spec-and-eval) https://git.io/v1n7F
<FromGitter>
<taylorfinnell> i wonder if the strange output is something on my end, or a bug in the formatter
<crystal-gh>
[crystal] asterite pushed 1 new commit to master: https://git.io/v1iu5
<crystal-gh>
crystal/master c5e1388 rogerdpack: make builds cross compile to correct target if using the llvm triple (--target) x86_64-apple-darwin16.1.0 which my sierra box with llvm 3.9 advertises
Ven has joined #crystal-lang
<crystal-gh>
[crystal] asterite pushed 1 new commit to master: https://git.io/v1iz9
<crystal-gh>
crystal/master 898a7ee Dylan Drop: Fixes #3547; as call with implicit receiver when invoked as a parameter to a method call
sp4rrow has joined #crystal-lang
<crystal-gh>
[crystal] asterite closed pull request #3671: Improve rehash when collision is high (master...optimize-rehash) https://git.io/v1oEO
bjz has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
soveran has quit [Remote host closed the connection]
soveran has joined #crystal-lang
soveran has quit [Remote host closed the connection]
soveran has joined #crystal-lang
soveran has joined #crystal-lang
soveran has quit [Changing host]
soveran has quit [Remote host closed the connection]