<Yxhuvud>
I got comfortable by doing advent of code. Too many searches and too many exercises on that silly language, but still good
bjz has joined #crystal-lang
<crystal-gh>
[crystal] MakeNowJust opened pull request #4017: Correct a doc comment for LiteralExpander of StringInterpolation (master...fix/crystal/string-interpolation-comment) https://git.io/vDVEm
soveran has quit [Remote host closed the connection]
soveran has joined #crystal-lang
soveran has joined #crystal-lang
soveran has quit [Changing host]
danielpclark has quit [Quit: Leaving]
soveran has quit [Remote host closed the connection]
bjz has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
mgarciaisaia has quit [Quit: Leaving.]
mgarciaisaia has joined #crystal-lang
Philpax has quit [Ping timeout: 276 seconds]
mgarciaisaia has quit [Ping timeout: 240 seconds]
Philpax has joined #crystal-lang
mgarciaisaia has joined #crystal-lang
yopp has quit [Ping timeout: 240 seconds]
skinkitten has quit [Ping timeout: 264 seconds]
skinkitten_ has quit [Ping timeout: 255 seconds]
_whitelogger has joined #crystal-lang
undy1ng has quit [Ping timeout: 255 seconds]
<FromGitter>
<spalladino> @kawikaconnell compiling the compiler might take up quite a lot of RAM. 2048 sounds enough, but if it's still complaining, then you might need to double it indeed. However, I'm just curious (maybe I missed a message earlier in the chat, sorry if that's the case), but why do you need to compile the whole compiler? Have you checked the usual installation methods for Ubuntu? https:/
<FromGitter>
<sdogruyol> @Rx14 yeah Kemal docs definitely has more room for improvements
mgarciaisaia has quit [Quit: Leaving.]
undy1ng has joined #crystal-lang
bjz has joined #crystal-lang
mgarciaisaia has joined #crystal-lang
Rinkana has joined #crystal-lang
bjz has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
bjz has joined #crystal-lang
soveran has joined #crystal-lang
mgarciaisaia1 has joined #crystal-lang
mgarciaisaia1 has quit [Client Quit]
mgarciaisaia has quit [Ping timeout: 256 seconds]
_whitelogger has joined #crystal-lang
Dreamer3 has joined #crystal-lang
Dreamer3 has quit [Ping timeout: 255 seconds]
Raimondii has joined #crystal-lang
Raimondi has quit [Ping timeout: 268 seconds]
Raimondii is now known as Raimondi
squeaky_pl has joined #crystal-lang
<squeaky_pl>
Hi, I just started playing with crystal lang a little. I played with Nim a little in the past. First thing I tried was compiling a "Hello world" and I looked through the generated ASM which was kind of big (almost 2MB). I also noticed that the resulting binary links to pcre. Is that the prelude "overhead"? How can I get a "minimal" binary with crystal?
<FromGitter>
<spalladino> Well, you can compile with `--prelude=empty`, but you won't even be able to do a `puts` in that case
<squeaky_pl>
spalladino, thanks, gonna look into that
<Yxhuvud>
squaky: how much do you get if you turn on optimizations?
<squeaky_pl>
Yxhuvud, it gets smaller from 1.7 MB to 1.2 MB
<Yxhuvud>
still a lot. I guess you link a lot statically then. What OS are you using?
<squeaky_pl>
Well I've seen the thread and also the tinycr challenge, I think this answers my question pretty well. I am not concerned about size, I was only curious
<squeaky_pl>
Yxhuvud, these are ASM output sizes
<squeaky_pl>
the binary is 373k in release mode after stripping
<squeaky_pl>
and I'm on Fedora
<crystal-gh>
[crystal] spalladino pushed 1 new commit to master: https://git.io/vDVbG
<crystal-gh>
crystal/master 102be34 TSUYUSATO Kitsune: Correct a doc comment for LiteralExpander of StringInterpolation...
<squeaky_pl>
Wow, it takes 70 seconds on my machine to compile the server example from the front page, is there anything I can do to make it go faster?
<squeaky_pl>
70 seconds in release mode
<FromGitter>
<greenbigfrog> Hi ⏎ What's the "best" way to close child processes, when the main process closes?
<FromGitter>
<spalladino> @squeaky_pl release builds *are* slow. May I ask how much it is taking for building in dev mode? I'm seeing 20s vs 5s in my machine for that example.
<squeaky_pl>
spallando it's 31.16
<FromGitter>
<spalladino> Still quite high
<squeaky_pl>
Does crystal do something like incremental compilation yet?
Dreamer3 has joined #crystal-lang
<squeaky_pl>
Also the server is not HTTP compliant, it fails under pipelining test
<squeaky_pl>
I mean it doesnt implement HTTP 1.1 correctly
<Yxhuvud>
well, while it should be supported, do notice that the browsers won't generally use pipelining anyhow.
<squeaky_pl>
Yes, but it's still a violation of HTTP
<squeaky_pl>
You cannot call it a compliant HTTP server
Dreamer3 has quit [Ping timeout: 240 seconds]
<squeaky_pl>
so if somebody puts that somewhere on the internet for a microservice for example and happens to get traffic from pipelining client he might get midly surprised
<squeaky_pl>
I mean it should not pipeline back responses but it shouldn't fail at least
<FromGitter>
<spalladino> Incremental compilation is in the roadmap, but still not supported
<squeaky_pl>
spalladino, impressive and interesting
<squeaky_pl>
I'm actually interested in HTTP in general and HTTP implementation guts
<Papierkorb>
squeaky_pl: it would surprise me though if the current HTTP/1.1 RFC would require pipelining support
<squeaky_pl>
Papierkorb, it does
<squeaky_pl>
it requires a server to not fail under pipelining clients
<Papierkorb>
Looking at RFC7230 §6.3.2 "Pipelining" ... couldn't they have chosen a more explicit wording?
<Papierkorb>
It says clients may pipeline, but nothing about the server behaviour in general, only that it again may run those in parallel
<squeaky_pl>
As far as I know and I talked a lot about it with Python community a server that does not handle pipelining is non conforming
<squeaky_pl>
Also a Wikipedia article on pipelining states that
<FromGitter>
<spalladino> I guess the first thing to do would be to clarify that in the docs at https://crystal-lang.org/api/0.20.5/HTTP/Server.html. @squeaky_pl have you spotted any other missing compliances that would be worthwhile adding? Would you like to submit a PR, or open an issue, to have that documented first? Then we can look into adding pipelining support. Or go full HTTP2 and finish what' missing from https://github.com/ysbaddaden/http2
Dreamer3 has joined #crystal-lang
<squeaky_pl>
spalladino, no, not yet :-) Just playing around. Indeed HTTP/2 multiplexing is far superior to pipelining but I don't have practical experience with that
<Papierkorb>
Does the server send a Date header btw in http/1.1? Does it allow keep-alive for 1.0?
<squeaky_pl>
You can do keep-alive with 1.0 and many clients work with that but it's outside HTTP/1.0
<squeaky_pl>
Papierkorb Not sure about the Date header, NodeJS and Go does send one by deault but you can disable it
<Papierkorb>
Don't remember if the Date header was a MUST or a SHOULD
<squeaky_pl>
A client that doesnt support HTTP/1.0 and receives keep-alive will just close the connection if it conforms
Dreamer3 has quit [Ping timeout: 245 seconds]
<Papierkorb>
sure, as long the default for 1.0 is no keep-alive that's fine
<Papierkorb>
otoh, anyone got statistics how many pure 1.0 clients are still out there?
<squeaky_pl>
Impressive work BTW
<squeaky_pl>
Papierkorb, I think not so many but as always ... you will get some clients written in old Java
<squeaky_pl>
browsers are never a problem, it starts when you need to plug several older systems together
<Papierkorb>
squeaky_pl: meant it as an prioritization. if keep-alive is not support atm for http/1.0 clients, then who cares really, people who want that have been using 1.1 clients for over a decade.
<squeaky_pl>
yes, of course
<squeaky_pl>
Conceptully implementing pipelining means maintaining a queue of incoming requests and only writing back to the wire in right order.
<squeaky_pl>
it's not so hard I would think and I might be interested in implementing it. It doesnt affect speed of non-pipelined flow in any way
<squeaky_pl>
you can also do optimizations there like using scatter/gatther IO to flush your buffers
<squeaky_pl>
you can easily do 1 million of RPS with these optimizations
<FromGitter>
<sdogruyol> @squeaky_pl just curious, why are you interested in Http clients that much :P
<FromGitter>
<sdogruyol> that looks interesting. Is that a web framework-ish?
<squeaky_pl>
sdogruyol, it's pretty bare bones but if you wanna discuss more there is #japronto
<squeaky_pl>
my interest in crystal stems from my disgust for C
<FromGitter>
<sdogruyol> that makes sense
<FromGitter>
<sdogruyol> BTW i also write a web framework (https://github.com/kemalcr/kemal). It's not barebones but more like Sinatra / Flask / Express
<FromGitter>
<sdogruyol> always looking for more performance OFC :P
<FromGitter>
<spalladino> @squeaky_pl adding pipelining sounds like a very nice contribution if you are interested, specially if it does not affect the non-pipelined flow. I'd advise to open an issue to check for acceptance before you start implementing, just to be sure we are not missing anything.
<squeaky_pl>
spalladino i think with Crystal's macro magic you can do it in a nice way and have perf in both cases, but I need to look more into Crystal and it's HTTP impl
bjz has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
Dreamer3 has joined #crystal-lang
Dreamer3 has quit [Ping timeout: 245 seconds]
<squeaky_pl>
how much of crystal is a string subset of Ruby?
<squeaky_pl>
I don't know Ruby myself
<squeaky_pl>
strict subset*
<FromGitter>
<spalladino> Syntactically speaking, there is quite some overlap. From the top of my head, main differences are type annotations (d'oh), a few renames such as `attr_accessor` to `property`, and several method aliases that are deliberately missing from crystal.
<FromGitter>
<spalladino> Semantically there are a few more, the `#[]` vs `#[]?` accessor is the first thing most people bump into when trying to translate directly from Ruby
<FromGitter>
<drosehn> A die-hard ruby programmer will repeatedly trip over differences. Some are pretty obviously needed (for instance, that you would sometimes need to add types in crystal, and you'd never need that in ruby). Some may seem odd at first, but happen due to other goals of crystal.
<FromGitter>
<drosehn> but there isn't any hard-and-fast rule which would accurately describe all the differences. crystal is very strongly inspired by ruby, but they're not trying to make a complete mirror of ruby.
<FromGitter>
<drosehn> I've written quite a lot of ruby in the last 15 years, although no really major projects. (certainly nothing close to the size of Rails!). I've come across a lot of little differences, but the two languages are close enough that it isn't too hard to switch back-and-forth between them.
<FromGitter>
<sdogruyol> @spalladino that book definitely needs more content :P
<FromGitter>
<drosehn> But I think that if you take some larger ruby program, see crystal for the first time and think "Ooo, I'll just convert this to crystal to get a huge performance boost", then all those little differences will really irritate you.
<FromGitter>
<drosehn> Also note that crystal is not at version 1.0 yet, and thus it's still willing and able to make significant changes. Such as dropping support for user-defined global variables (such as `$my_option`).
<FromGitter>
<spalladino> I know some people who have had good experiences with "translating" code from Ruby, others the complete opposite. I guess it depends a lot on the style of the program.
<FromGitter>
<sdogruyol> IMHO Crystal won't make Rails developers happy for now
<FromGitter>
<sdogruyol> On the other hand Ruby developers are mostly falling in love <3
<FromGitter>
<spalladino> Personally I like that Crystal doesn't allow much room for "Railsy" code, but that's for another discussion
<squeaky_pl>
So why I am really asking about that is jumping into Crystal for me would be a lot of write - compile iterations and since compiler seems slow that might get annoying. So I was thinking maybe I could use Ruby interpreter to get the logic right first
<squeaky_pl>
Or how do you overcome this normally, do you need a faster machine?
<FromGitter>
<drosehn> I do that for some things. or I'll use the `crystal play` support, which is very nice for small experiments.
<FromGitter>
<spalladino> It's up to you, but I'd strongly advise against it. The differences in Ruby are enough for the translation afterwards to be a pain, and sincerely I have never had many issues with compiler times (specially since you'll be working in dev and not release mode usually)
<FromGitter>
<sdogruyol> Catching up with Crystal release version numbers :P
<FromGitter>
<drosehn> I should note that what I'll use ruby for is for very small experiments, on the order of a few lines typed into `irb`. I would *not* write an entire method in ruby and then convert it to crystal.
<squeaky_pl>
so with PyPy you can use RPython that just works like normal Python and it's strict subset and then you translate that to C
<FromGitter>
<drosehn> And I do that because I already have a lot of experience with ruby, so I'm just plain used to doing that. If you don't already know ruby, I wouldn't spend any time testing any crystal code in ruby.
<squeaky_pl>
that's where I got the idea of using Ruby interpreter...
<jeromegn>
@spalladino glad to have written something that resonated with someone :D
bjz has joined #crystal-lang
<FromGitter>
<drosehn> The crystal compiler is not all that slow. Just get used to writing short examples to test out something that you're learning. And once you get that working, then add it to your larger crystal-based project.
<FromGitter>
<drosehn> ... and stay away from "release" compiles until you're really at the stage where you have a complete product that you're ready to release!
<squeaky_pl>
yeah, I got you, it boils down to writing unit tests basically, if you can unit test your code it's gonna be fine
<FromGitter>
<drosehn> Well, you don't need to do full unit-test programming, although that has additional benefits (especially if you are tackling a large project).
<squeaky_pl>
I have to admit I like Crystal more than Nim after toying with it a little
<squeaky_pl>
everything written in the gitbook fits my brain
bjz has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<FromGitter>
<drosehn> For some things I write, I really need to create a compiled-object and not depend on some interpreter environment. For those projects, I feel that crystal is the most promising language that I've tried. Or at least it is for me, as someone who is very comfortable writing ruby.
<squeaky_pl>
is there any guide on using gdb with crystal?
<squeaky_pl>
or should I use something else
<squeaky_pl>
putting a breakpoint seemed to work but I cannot print anything out
<squeaky_pl>
ah I got it
<squeaky_pl>
I need -d flag
<Yxhuvud>
regarding compilation speed, I wonder if that is a regression. I don't recall there was much complaining about it a few versions back.
<squeaky_pl>
Yxhuvud, I am not a 7 year old macbook pro
<squeaky_pl>
I am on *
<Yxhuvud>
well, you are not the only one that noticed kemal being a bit slow to compile lately.
<FromGitter>
<drosehn> There have been a few different people who have mentioned that the most recent versions of crystal seem to be slower. Not enough to get anyone fired up, but enough for them to make a comment about it. There is the recent issue #4015 which might be the culprit for some of the slowdowns.
<Rinkana>
How can i get only the text inside a XML::Node? Both .text and .inner_text return the full content (including childs...)
<Rinkana>
It seems that xpath returns an XML::Node and as it's children it has Text nodes
mgarciaisaia has quit [Ping timeout: 276 seconds]
mgarciaisaia has joined #crystal-lang
Dreamer3 has joined #crystal-lang
Dreamer3 has quit [Ping timeout: 260 seconds]
Dreamer3 has joined #crystal-lang
<bmcginty>
rinkana: right. so elementNode.children[0] will return the first text node inside elementNode.
<Rinkana>
Yeah, but the thing is that it does not have to have a text node as first element. And it can have multiple
<Rinkana>
I've fixed it by doing: xml_node.children.select(&.text?).map(&.to_s).join("")
<bmcginty>
Rinkana: okay. so you want what, a method to return all the text nodes at one level below a selected node, concatinated...?
<bmcginty>
Rinkana: ah. sweet. thanks.
<Rinkana>
However i do find it odd that .text and .inner_text do exactly the same as .content
<bmcginty>
Rinkana: I wonder if that should be put up as a PR? not sure what it's be called, though?
<Rinkana>
bmcginty: i'd put it in place of the current .inner_text
Dreamer3 has quit [Ping timeout: 240 seconds]
<Rinkana>
However, what to do with spaces? Sometimes text nodes can be just a space.
<bmcginty>
Rinkana: if you want to do a PR for it, feel free. if not, if I can take that code you just posted and add your nick, I'll put one up within the week.
<bmcginty>
Rinkana: so maybe a normalized_text option? strip all multiple spaces to a single space?
<Rinkana>
Call the .squeeze?
<bmcginty>
Rinkana: Can you share what you're using this on? I'd like to look at a couple other language libs and see their differences as well.
Dreamer3 has joined #crystal-lang
<Rinkana>
What do you want to know?
<Rinkana>
I'm running crystal 0.20.5 on mac. I came across this issue because .text in Ruby and .text in Crystal do not yield the same result.
<bmcginty>
Rinkana: I'll go through rubys xml stuf then. Looking at python, which was my go to before crystal, it just pulls text from the first child node, _if it's a text node, if I'm reading correctly.
<bmcginty>
Rinkana: I shouldn't be using a python mindset anywhere near crystal anyhow.
<Rinkana>
haha, it does work fine for the most part tho
bjz has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<RX14>
i find it weird that we get all these performance complaints recently
<RX14>
something's definitely going on
<RX14>
considering I can compile the whole compiler (without release) faster than squeaky_pl said his simple webapp took
<Papierkorb>
RX14: I saw two people in here saying that, and both used Mac. Is the brew build broken? Or is it an issue with the recent performance-fix for Linux systems 'breaking' on the mac?
<RX14>
i have no idea
<RX14>
whatever it is it doesn't seen to affect linux
<RX14>
or actually i think
<RX14>
maybe I found someone on linux for which it did...
<Papierkorb>
Could `crystal version` maybe output the platform (including if it's brew-build or such, if possible), so when this happens one can just ask for the version output?
<Papierkorb>
or is `uname -a` a thing on mac os?
<RX14>
i find it interesting that crystal's barely-optimized http server beats go
<Papierkorb>
I don't know what it is exactly, but something in the compiler - Crystal itself or LLVM - does an excellent job
<RX14>
it allocates a whole new array!
<RX14>
just to read the first line
<RX14>
and it beats go
<RX14>
with unneccesary allocations on the hot path
<RX14>
it's weird
<Papierkorb>
I'm currently working on a lightning-fast rpc lib (cause I need it for my mmo stuff), and the only real bottleneck is memory allocation during decoding :)
<RX14>
yeah memory allocation is the number 1 performance consideration
<Papierkorb>
So if one wants to go much faster, while compiler improvements are always great, the GC looks to be a promising target too
<RX14>
apart from bad algorithms of course
<RX14>
well reducing allocations is usually the way to go
<RX14>
and 0-copy
<Papierkorb>
I'm atm hitting 100M/s serialisations for a simple data structure (an 3 element `Array(Int32)`) and 25M/s deserialisations of the same structure
<Papierkorb>
Bottleneck for serialisation boils down to memcpy(), yes
<RX14>
can you show me the code?
<RX14>
i'll have a look through
<RX14>
measure in MB/s for serialization/deserialization
<Papierkorb>
I'll put it up on github later when I'm happy with it
<RX14>
it's usually better
<Papierkorb>
Not sure in this particular case. It's designed to be used between gateway server to the backend servers and if I'm crazy enough, for comms between a backend server and its hot-standby counterpart. So I have many small-ish data structures, many of which are actually static-sized, the real "issue" is that there'd be a significant packet amount if few thousand clients are connected at once.
<Papierkorb>
Well let's be honest: I'm designing that thing for that, but I won't use it for it. Though I want to design it as real-world-y as possible for the scale of the original
<Papierkorb>
RX14: Does MiB/s still apply for that use-case?
<RX14>
find out :)
<RX14>
record in ips, and record serialized size
<RX14>
plot at different sizes and etc.
<RX14>
place on google doc
<RX14>
graph it up
<Papierkorb>
Good idea
<RX14>
i want to make a GUI microbenchmarking framework inside crystal play
<RX14>
where you can parameterize benchmarks etc.
<RX14>
graph them
<Papierkorb>
My test-case generates ~1.5GiB/s. Will produce a more throughout thing later (tomorrow or on monday?)
<RX14>
memory bandwidth is about 10GiB/s iirc
<Papierkorb>
The real bottleneck *will* be the network, so in that terms, it would technically saturate a 12GBit/s connection .. which doesn't exist afaik, but a 10GBits connection would be saturated :)
<RX14>
40Gbit connectios are happening
<RX14>
so 5GB/s so
<RX14>
like 4 cores
<Papierkorb>
Sounds somewhat reasonable?
<RX14>
very reasonable
<RX14>
thats "lets do userspace tcp" territory
<Papierkorb>
Oh I remember reading about Cloudflare (I think?) talking about that in one of their blogs. was a fine read
<Papierkorb>
I think they said something along of "UNIX was never built for bandwith consumption, but for routing. [..] For bandwith, you'd design it totally different" I still wonder how such a bandwith-oriented architecture would look like
<Papierkorb>
Saying "oh lets throw out iptables" doesn't cut it I guess :)
<RX14>
very direct write path from application to NIC
<Papierkorb>
And probably listening for incoming data through IRQs. Sounds like a use-case for a Realtime OS
<RX14>
ya
snsei has joined #crystal-lang
snsei has quit [Read error: No route to host]
snsei has joined #crystal-lang
<FromGitter>
<zatherz> is it intended that you can't use macros in the generic part of the type
<FromGitter>
<zatherz> like `Test({{"String".id}})`
<FromGitter>
<zatherz> `expecting token 'CONST', not '{{'`
<BlaXpirit>
zatherz, it makes sense in some way. when a compiletime expression is expected, these brackets are not expected
<BlaXpirit>
you can move the brackets outwards, should work
<BlaXpirit>
it's probably not intended. i'm just making up excuses :|
<Papierkorb>
I found something akin to that: `StaticArray(UInt8, sizeof(typeof(Foo)).new` fails with a message that it didn't expect an Int32, but an integer value for the second generic argument
<Papierkorb>
Found another (faster) way of expressing what I actually needed, but still expected that one to work
<BlaXpirit>
nah that one just makes sense
<BlaXpirit>
it's to prevent recursion in type resolution
<BlaXpirit>
but annoying for sure
<Papierkorb>
It can happily fail with an error if it can't deduce the real type, but for a simple case it should work
<RX14>
@zatherz as always the macro has to parse on it's own, you need to surround a whole method/class with {% begin%}/{% end %} to be able to template such places
<BlaXpirit>
RX14, i don't think this is the underlying reason
Rinkana has quit [Ping timeout: 260 seconds]
<RX14>
i do
<RX14>
prove me wrong :)
soveran_ has quit [Remote host closed the connection]
soveran has joined #crystal-lang
snsei has quit [Remote host closed the connection]