<bmcginty>
Anyone here done speed tests for serialization/deserialization of data? I've got about 400 GB to transform from JSON into something that I can go through as quickly as possible to do some calculations. This is definitely a DB task, but I kind of want to try a different method...just don't want to have to parse the json into something, only to find out something better exists.
tzekid has quit [Ping timeout: 246 seconds]
tzekid has joined #crystal-lang
<FromGitter>
<codenoid> ok, i have a question, can i scrap full (100% loaded) web data (loaded by ajax) with just using crystal and cr html parser lib ? (without web driver)
Dreamer3_ has quit [Quit: Leaving...]
greengriminal has joined #crystal-lang
rohitpaulk has joined #crystal-lang
greengriminal has quit [Client Quit]
deep-book-gk_ has joined #crystal-lang
deep-book-gk_ has left #crystal-lang [#crystal-lang]
Dreamer3_ has joined #crystal-lang
<oprypin>
codenoid, usually yes
Ven has joined #crystal-lang
Ven is now known as Guest24574
<FromGitter>
<codenoid> how ? pwizz tell me
<FromGitter>
<codenoid> 1) gimme example 😄
Guest24574 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Ven_ has joined #crystal-lang
Ven_ has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<FromGitter>
<konovod> @bmcginty if you don't need to store structure with data, binary dump is a fastest, so i suggest https://github.com/Papierkorb/cannon
sooli has joined #crystal-lang
mark_66 has joined #crystal-lang
sooli has quit [Quit: sooli]
snsei has joined #crystal-lang
<FromGitter>
<konovod> On the other hand, binary dump is useless for 400gb data as you have to seek through it, not load. So scratch that.
sooli has joined #crystal-lang
snsei has quit [Remote host closed the connection]
sooli has quit [Ping timeout: 258 seconds]
saadq has quit [Read error: Connection reset by peer]
sooli has joined #crystal-lang
saadq has joined #crystal-lang
sooli has quit [Client Quit]
<FromGitter>
<codenoid> other way to download file in crystal without use unix cmd ( ` `wget myfile.jpg` ` )
Raimondii has joined #crystal-lang
hightower has joined #crystal-lang
Raimondi has quit [Ping timeout: 268 seconds]
Raimondii is now known as Raimondi
_whitelogger has joined #crystal-lang
_whitelogger_ has quit [Remote host closed the connection]
<FromGitter>
<codenoid> hi, how to activate cookie in HTTP::Client
sz0 has joined #crystal-lang
<Papierkorb>
bmcginty: That is indeed a task for a database. You could build something sophisticated, but really, it would end up as kind of database. Postgres is a fantastic database, and should handle that load just fine.
rohitpaulk has joined #crystal-lang
Qchmqs__ has joined #crystal-lang
<crystal-gh>
[crystal] mverzilli closed pull request #4735: Tuple: support negative index (master...feature/tuple/support-negative-index) https://git.io/v7kYm
<crystal-gh>
[crystal] MakeNowJust opened pull request #4748: Use {% begin %} instead of {% if true %} (master...fix/use-macro-begin-instead-if-true) https://git.io/v73VN
snsei has quit [Remote host closed the connection]
snsei has joined #crystal-lang
mark_66 has quit [Remote host closed the connection]
sz0 has quit [Quit: Connection closed for inactivity]
<adam12>
Does anybody know where/how freebsd status left off? I see a binary package on 0.19 release page, which works! but using 0.19 to compile 0.20 fails :\
sz0 has joined #crystal-lang
snsei has quit [Remote host closed the connection]
<RX14>
we theoretically have freebsd support
<RX14>
how does the build fail adam12?
<adam12>
RX14: Let me see if I can rerun it again.
Qchmqs__ has quit [Ping timeout: 240 seconds]
<crystal-gh>
[crystal] jhass closed pull request #4748: Use {% begin %} instead of {% if true %} (master...fix/use-macro-begin-instead-if-true) https://git.io/v73VN
<adam12>
RX14: I wonder if it's related to LLVM version. I'm using LLVM 4.0 to build 0.20, but I remember something about only 3.6 being supported somewhere?
<adam12>
RX14: Let me try downgrading LLVM to 3.9
<RX14>
no
<RX14>
4.0 is supported
<RX14>
i need an error message to tell you anything
<adam12>
I know. I can't copy/paste out of this VM, so I was trying not to waste your time.
<adam12>
I piped to a text file but LLVM outputs those raw colour codes, and I'm not sure how to strip them to have an elegant error message.
<RX14>
the first thing to do with a VM is set up ssh to it
alibby has joined #crystal-lang
<adam12>
RX14: I might just do that.
<RX14>
so you can close the GUI and leave it in the background
<adam12>
Should I build incremental versions, or just skip from 0.19 binary to latest release?
<adam12>
RX14: So 0.20 builds with LLVM3.9, no errors.
<RX14>
oh uhh maybe 0.20 doesn't support llvm 4
<RX14>
i forgot you were using old version
<RX14>
i forgot you were using old versions
<adam12>
Yeah - should I build one, and then use that built one for the next version, starting 0.19?
<adam12>
Or do you think 0.19 building 0.23.1 should just work?
<RX14>
if you have linux you can just cross-compile
<adam12>
I do. I was curious what it would take to package crystal in freebsd ports.. and if it was even possible.
<RX14>
i don't know
<RX14>
bootstrapped packages are always a bit weird
<RX14>
look if there's a rust/gcc/go port and how they appear to do it
<hightower2>
Are you folks aware that the Release Notes displayed at the bottom of https://crystal-lang.org/ are out of date? The last ones shown are for Crystal 0.20.0
<FromGitter>
<mverzilli> yes, those are more "release note blogposts" than release notes
<FromGitter>
<mverzilli> but it is confusing, I noticed them the other day and thought the same
hightower2 has quit [Ping timeout: 255 seconds]
<FromGitter>
<oprypin> me too thanks
catfuneral[m] has left #crystal-lang ["User left"]