ChanServ changed the topic of #crystal-lang to: The Crystal programming language | http://crystal-lang.org | Crystal 0.20.5 | Fund Crystal's development: http://is.gd/X7PRtI | Paste > 3 lines of text to https://gist.github.com | GH: https://github.com/crystal-lang/crystal | Docs: http://crystal-lang.org/docs/ | API: http://crystal-lang.org/api/ | Logs: http://irclog.whitequark.org/crystal-lang
olbat has quit [Ping timeout: 240 seconds]
olbat has joined #crystal-lang
akai-iro has joined #crystal-lang
akai-iro is now known as akaiiro
dannyAAM has quit [Ping timeout: 264 seconds]
dannyAAM has joined #crystal-lang
snsei has joined #crystal-lang
snsei has quit [Ping timeout: 255 seconds]
mgarciaisaia has quit [Quit: Leaving.]
soveran has quit [Remote host closed the connection]
soveran has joined #crystal-lang
soveran has quit [Changing host]
soveran has joined #crystal-lang
soveran has quit [Remote host closed the connection]
squeaky_ has quit [Ping timeout: 256 seconds]
snsei has joined #crystal-lang
snsei has quit [Ping timeout: 276 seconds]
wdfwefewvfgew has joined #crystal-lang
wdfwefewvfgew has left #crystal-lang [#crystal-lang]
<marciogm> Where i can find the implementation for @[Include()] ?
akaiiro has quit [Ping timeout: 256 seconds]
bjz has joined #crystal-lang
panderssen has joined #crystal-lang
<panderssen> could anyone help a newcomer to crystal find the issue in this simple code?
<panderssen> the error i get is "Repeated allocation of very large block"
<panderssen> i am opening dozens of large files, each aroudn 30MB, but after some light search/replace then they are written to disk and on to the next
<panderssen> is there a way to 'deallocate' or 'garbage-collect' them? (i would have thought that the loop ending would do so automatically...)
<panderssen> and eventually the script will eat up all available RAM and crash
bjz has quit [Read error: Connection reset by peer]
bjz has joined #crystal-lang
panderssen has quit [Quit: Page closed]
Ven has joined #crystal-lang
soveran has joined #crystal-lang
soveran has joined #crystal-lang
soveran has quit [Changing host]
mark_66 has joined #crystal-lang
<FromGitter> <drosehn> It's 3am for me right now, but I might look at that tomorrow. But one question that I have is why do you have to read in the entire file all at once, instead of doing it a line-at-a-time? Is the gsub matching a multi-line pattern?
<FromGitter> <drosehn> It *might* help to add a `xml = nil` or `xml = ""` after the `puts`, before it loops back up. But you shouldn't need to do that.
bjz has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
bjz has joined #crystal-lang
Ven has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Ven has joined #crystal-lang
badeball_ is now known as badeball
gloscombe has joined #crystal-lang
Ven has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
havenwood has quit [Quit: mIRC 5.91 (16 bit) for Microsoft © Windows For WorkGroups 3.11®]
havenwood has joined #crystal-lang
havenwood has quit [Changing host]
havenwood has joined #crystal-lang
bjz has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
bjz has joined #crystal-lang
squeaky_ has joined #crystal-lang
Ven has joined #crystal-lang
sz0 has joined #crystal-lang
Raimondii has joined #crystal-lang
Raimondi has quit [Ping timeout: 268 seconds]
Raimondii is now known as Raimondi
bjz has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
bjz has joined #crystal-lang
bjz_ has joined #crystal-lang
bjz has quit [Ping timeout: 255 seconds]
Ven has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
crystal-lang119 has joined #crystal-lang
<crystal-lang119> hello everyone
crystal-lang119 has quit [Client Quit]
<FromGitter> <sdogruyol> @RX14 i feel you bro
<FromGitter> <sdogruyol> @crystal-lang119 welcome
Rinkana has joined #crystal-lang
bjz has joined #crystal-lang
bjz_ has quit [Ping timeout: 240 seconds]
bjz_ has joined #crystal-lang
<FromGitter> <crisward> Anyone got examples of using Process.run
bjz has quit [Ping timeout: 240 seconds]
<RX14> @crisward are you haveing problems with it?
<RX14> what do you want to do
<FromGitter> <crisward> need to run `du /dir/ -s` and read output. Trying to get the size of a folder.
<FromGitter> <crisward> Wasn't sure how to access output
<RX14> if you look at the docs it says that stdout has a "stdio" type
<RX14> if you click the stdio type it says what the cvalues can be and what they mean
<RX14> in this case you'd want `output: io`
<RX14> where io is some memoryIO you created
<FromGitter> <crisward> ok
<FromGitter> <crisward> ```code paste, see link``` ⏎ ⏎ seems to work, thanks. [https://gitter.im/crystal-lang/crystal?at=58a19a5faa800ee52c8b9119]
bjz has joined #crystal-lang
bjz_ has quit [Ping timeout: 255 seconds]
<FromGitter> <sdogruyol> Cheers @crisward
sz0 has quit [Quit: Connection closed for inactivity]
bjz has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
bjz has joined #crystal-lang
undy1ng has quit [Ping timeout: 264 seconds]
bjz has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
mgarciaisaia has joined #crystal-lang
mgarciaisaia has quit [Client Quit]
mgarciaisaia has joined #crystal-lang
soveran has quit [Remote host closed the connection]
Yxhvd has joined #crystal-lang
endou_ has joined #crystal-lang
A124 has quit [Disconnected by services]
A124 has joined #crystal-lang
stnly has joined #crystal-lang
endou has quit [Write error: Broken pipe]
Yxhuvud has quit [Write error: Broken pipe]
stnly_ has quit [Remote host closed the connection]
soveran has joined #crystal-lang
soveran has joined #crystal-lang
soveran has quit [Changing host]
soveran has quit [Remote host closed the connection]
soveran has joined #crystal-lang
soveran has quit [Remote host closed the connection]
A124 has quit [Quit: '']
soveran has joined #crystal-lang
soveran has joined #crystal-lang
soveran has quit [Changing host]
A124 has joined #crystal-lang
e_ has joined #crystal-lang
e_ has quit [Ping timeout: 260 seconds]
soveran has quit [Remote host closed the connection]
soveran has joined #crystal-lang
snsei has joined #crystal-lang
pabs_ is now known as pabs
soveran has quit [Remote host closed the connection]
<crystal-gh> [crystal] asterite pushed 3 new commits to master: https://git.io/vDoDI
<crystal-gh> crystal/master 009d9bc Ary Borenszweig: Compiler: use a different LLVM::Context for each LLVM::Module
<crystal-gh> crystal/master 193225d Ary Borenszweig: LLVM: remove global context and related functions...
<crystal-gh> crystal/master 4c6708c Ary Borenszweig: LLVM: bind Context functions, and apply a general refactor
snsei has quit [Remote host closed the connection]
<Rinkana> Can anyone give me a quick rundown on the status of shards?
<RX14> how many there are? how they work?
<RX14> what do you want to nkow
<Rinkana> Mainly the status on the repo functionality
<Rinkana> If i somehow already can define dependencies that are the downloaded from GitHub
<RX14> you mean depend on a github repo?
<RX14> thats easy
<Rinkana> Yeah
<Rinkana> Ah thanks, last time i checked it was very much WIP
<RX14> there's a SPEC.md there too
<RX14> if you want the entire shards.yml story
<Papierkorb> Is there no SHM support in the stdlib?
<RX14> Papierkorb, like linux shared memory?
<Papierkorb> yes
<RX14> no i don't think so
<RX14> shouldn't be hard to bind
Rinkana has quit [Quit: Page closed]
<crystal-gh> [crystal] asterite pushed 1 new commit to master: https://git.io/vDo5y
<crystal-gh> crystal/master 4175143 Ary Borenszweig: LLVM: `get_module_context` and `get_global_parent` were available since LLVM 3.5.0
mark_66 has quit [Remote host closed the connection]
<crystal-gh> [crystal] asterite pushed 1 new commit to master: https://git.io/vDob1
<crystal-gh> crystal/master c60df09 Ary Borenszweig: LLVM: `get_module_identifer` and `set_module_identifier` were introduced in LLVM 3.9
pduncan has joined #crystal-lang
Yxhvd has quit [Remote host closed the connection]
Yxhuvud has joined #crystal-lang
<FromGitter> <chyzwar> @RX14 yesterday this bug about module ale logger.formatter this is know issue or I should raise issue in crystal repo?
<FromGitter> <chyzwar> I also have another one when rescue do not catch exception, https://carc.in/#/r/1msm
<travis-ci> crystal-lang/crystal#c60df09 (master - LLVM: `get_module_identifer` and `set_module_identifier` were introduced in LLVM 3.9): The build passed. https://travis-ci.org/crystal-lang/crystal/builds/201181725
soveran has joined #crystal-lang
soveran has joined #crystal-lang
soveran has quit [Changing host]
<crystal-gh> [crystal] Rinkana closed pull request #3111: [WIP] HTTP::Client middleware (master...http_client_middleware) https://git.io/v6seP
Rinkana has joined #crystal-lang
Raimondi has quit [Ping timeout: 268 seconds]
gloscombe has quit [Remote host closed the connection]
<RX14> @chyzwar yesterday's bug should be on the issue tracker
<FromGitter> <asterite> And I added one for the above: https://github.com/crystal-lang/crystal/issues/4030
<RX14> nice
<RX14> @asterite this was the other bug: https://carc.in/#/r/1mon
<RX14> expected @test to equal 5
<crystal-gh> [crystal] spalladino closed pull request #4025: Adds includes? macro method to string, symbol and macroid. (master...feature/macro_includes) https://git.io/vDw99
bew78 has joined #crystal-lang
soveran has quit [Remote host closed the connection]
soveran has joined #crystal-lang
<crystal-gh> [crystal] spalladino closed pull request #4024: Support "View source" links for GitLab projects (master...gitlab-view-source) https://git.io/vDw9J
<FromGitter> <sdogruyol> @RX14 seems like rust is moving off from their own CI to Travis and Appveyor https://twitter.com/rustlang/status/831219313189089280
soveran has quit [Remote host closed the connection]
<RX14> that seems counter-productive to testing on non-x86 architectures
<FromGitter> <sdogruyol> not sure what their reasoning was
soveran has joined #crystal-lang
soveran has joined #crystal-lang
soveran has quit [Changing host]
<RX14> i'm sure i'll be able to find the reasoning after some searching
<RX14> it seems their reasoning was that changing the CI config is hard
<RX14> which Jenkinsfile solves
<RX14> and also that the infastructure is fragile and falls over often
<RX14> which I hope isn't the case due to the maturity of the jenkins ecosystem
<RX14> but we'll see
<FromGitter> <sdogruyol> curious why they built their own ci in first place instead of using jenkins e.g
<RX14> because it was originally a mozilla project
<RX14> and probably wasn't on github
<RX14> or something along those lines
<FromGitter> <sdogruyol> guess they couldnt maintain it
<FromGitter> <sdogruyol> nor anyone wanted to
bjz has joined #crystal-lang
<bew78> :q
bew78 has quit [Quit: WeeChat 1.7]
soveran has quit [Remote host closed the connection]
Raimondi has joined #crystal-lang
soveran has joined #crystal-lang
soveran has joined #crystal-lang
soveran has quit [Changing host]
bjz has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
bjz has joined #crystal-lang
<travis-ci> crystal-lang/crystal#69cd7c8 (master - add "View source" links to GitLab): The build passed. https://travis-ci.org/crystal-lang/crystal/builds/201241824
soveran has quit [Remote host closed the connection]
bjz has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
Rinkana has quit [Ping timeout: 260 seconds]
soveran has joined #crystal-lang
soveran has joined #crystal-lang
soveran has quit [Changing host]
soveran has quit [Ping timeout: 252 seconds]
soveran has joined #crystal-lang
soveran has joined #crystal-lang
soveran has quit [Changing host]
<Papierkorb> As it's quiet in here right now, I'll take this opportunity to shamelessly plug my new data serialization and RPC shard `cannon`: https://github.com/Papierkorb/cannon
<RX14> Papierkorb, 5 cycles per to encode that array?
<RX14> that's insane
<Papierkorb> A simple Array(Int32) is nothing else than a Slice, so what you see there is memcpy() overhead
<Papierkorb> But thanks for the compliment ^^
<Papierkorb> I can still improve the slice decoding, atm it's doing useless memory zeroing before I copy stuff into it
<RX14> afaik libGC does memory zeroing
<Papierkorb> > ping 63.52k ( 15.74µs) (± 4.34%) fastest
<RX14> ?
<Papierkorb> That's the RPC mechanism doing pings like crazy. Wondering if I can get it any faster without sacrificing too much
<RX14> pings?
<Papierkorb> Look into samples/rpc/client.cr
<RX14> 63.52k?
<RX14> isn't that a bit slow
<RX14> crystal's HTTP is faster
<Papierkorb> Was around 26k/s ish last time I checked on my machine
<RX14> what's your specs Papierkorb
<Papierkorb> The RPC stuff also uses Fibers, or rather, 1 - 2 per call
<RX14> oh
<Papierkorb> Intel(R) Core(TM) i5-6600K CPU @ 3.50GHz on x64 ArchLinux
<RX14> huh
<RX14> HTTP is 100k rps for me
<Papierkorb> So I guess it's basically benchmarking the scheduler :)
<Papierkorb> using the hello world server from the page?
<RX14> yeah
<RX14> about 90k
<Papierkorb> Using ab (apache bench), 1M requests 1k concurrent, I get 51623.32 [#/sec] (mean)
<Papierkorb> 100 concurrent requests and it's 57690.91 [#/sec] (mean)
<RX14> with 1000 concurrent it's 70k for me
<RX14> i'm using wrk
<RX14> wrk -c 13 -t 2 http://localhost:8080/
<RX14> 13 connections 2 threads
<Papierkorb> what's the packet name?
<RX14> wrk
<Papierkorb> ah in AUR
<RX14> yes
<Papierkorb> Requests/sec: 182769.02
<RX14> what
mgarciaisaia has quit [Quit: Leaving.]
<Papierkorb> Okay, ab sucks
<RX14> how do you get 182k
<Papierkorb> I used your exact wrt command
soveran has quit [Remote host closed the connection]
<RX14> the only applicable difference is CPU generation
<Papierkorb> Is that DDR4 at work?
<Papierkorb> More recent RAM is actually making a difference at once? that's something new.
<RX14> i've gotten desperate
<RX14> using --mcpu=ivybridge
<RX14> great
<RX14> it's now worse
<RX14> lol
<RX14> huh
<RX14> no
<RX14> it's crystal master thats the problem
<RX14> weird
<RX14> as llvm does 90% of the work
<RX14> well
<RX14> 99% of the optimization
<RX14> weird
<RX14> wait a sec
soveran has joined #crystal-lang
<RX14> the cache of object files doesn't include --mcpu options then I guess
<RX14> oh I guess it does?
<RX14> nvm
<Papierkorb> RX14: Any idea how to trace a fiber deadlock which may or may not happen in the scheduler?
<RX14> attempt to reduce with as few fibers as possible
<RX14> and then puts debug
<RX14> the alternative is to cry I think
<Papierkorb> Have to test it, but all spawn{} heavy servers I write have had issues with not receiving any data, or not even accepting a tcp socket anymore at some point at random intervals, while those without fibers work just fine
<RX14> i've had probalems with camo.cr just failing
<RX14> just it refuses to do TCP connections outwards
<RX14> and it just hangs
<RX14> until the socket data timeout triggers
<RX14> so all the response times are multiples of 10 seconds
<Papierkorb> did you mess with #sync= or #tcp_nodelay ?
<RX14> no
<Papierkorb> RX14: Wrote a ping/pong test: https://gist.github.com/Papierkorb/f253edb22df08a13e6d8cba9f44cbc00 I'm only getting 115k round-trips per second. What's wrong with the code?
<RX14> it's only using a single fiber
squeaky_ has quit [Ping timeout: 240 seconds]
<RX14> creating a pool of 10 fibers to do the requests would be very helpful in boosting that
<RX14> same on the server
<RX14> it's limited by not being concurrent
<Papierkorb> the pool on the server doing what?
<RX14> uhh
<RX14> idk
<RX14> calling the accept loop
<RX14> I guess
<RX14> i guess multiple fibers can accept
<Papierkorb> I added a spawn around the loop{}, which somewhat resembles what I usually do
<Papierkorb> That is, a fiber accepts and spawns a fiber per incoming connection, reconnects per second isn't that important to me
sz0 has joined #crystal-lang
<Papierkorb> Updated the gist, the server.cr now somewhat resembles what I do in the RPC stuff: One accept fiber, a fiber per connection, and for each fully read incoming request, spawn a fiber doing the calculations (or nothing in this case), and writes the response out
<Papierkorb> That spawn galore costs me 10k requests/sec, so I now get 105k requests/sec. Which as a price on its own is acceptable