<FromGitter>
<watzon> This library is a very interesting challenge. I keep running into hangups.
<FromGitter>
<watzon> Right now I'm having trouble sending an asynchronous request and waiting for a response without stopping the loop that's checking for responses
<FromGitter>
<watzon> Basically I have an update loop running in the background, and that loop is constantly checking update handlers and sending responses to them if they match. Then I have the broadcast method which basically needs to send an asynchronous query and then listen for and return the result.
<FromGitter>
<watzon> It's proving to be an enormous pain in the ass
<FromGitter>
<whidbey> @watzon which ide are you using
<FromGitter>
<watzon> I'm using VS Code
<FromGitter>
<whidbey> is that good? using scry?
<FromGitter>
<Blacksmoke16> Hmm
<FromGitter>
<Blacksmoke16> Spawn with a channel and select?
<FromGitter>
<whidbey> could it hint class/method on serveral cr files?
<FromGitter>
<watzon> Is there documentation on select?
<FromGitter>
<watzon> @whidbey what do you mean? Currently there isn't a way to get good intellisense with Crystal, but the plugin does highlighting, formatting, and supports Scry
<FromGitter>
<Blacksmoke16> dunno how to use it tho
<FromGitter>
<watzon> Hmmm
<FromGitter>
<watzon> Well this `Channel.non_blocking_select(channel.receive_select_action)` works and doesn't block the thread, but it returns `{1, Channel::NotReady()}` rather than the value I'm after
f1reflyylmao has joined #crystal-lang
<FromGitter>
<Blacksmoke16> what value are you expecting?
<FromGitter>
<Blacksmoke16> id imagine you'd want to go back to it every so often to see if it has a value yet
f1refly has quit [Ping timeout: 245 seconds]
<FromGitter>
<watzon> Yeah that's the trick, is doing that in a way that won't block execution of the other thread
<FromGitter>
<watzon> Fuck concurrency
<FromGitter>
<Blacksmoke16> `extra = Random.new.hex(16)` the default is 16 btw
<FromGitter>
<watzon> Yeah I know haha. I had it set to 32 at first.
<FromGitter>
<Blacksmoke16> yea im not experienced enough in it to have a suggestion
<FromGitter>
<watzon> I just wish I could visualize what's happening with the fibers
ishahnaz has joined #crystal-lang
ishahnaz has quit []
duane has quit [Ping timeout: 240 seconds]
ht_ has joined #crystal-lang
ur5us has quit [Ping timeout: 245 seconds]
return0e_ has joined #crystal-lang
<FromGitter>
<watzon> I hate that `select` is completely undocumented. Doesn't show up in the API docs or the book.
return0e has quit [Ping timeout: 265 seconds]
<FromGitter>
<watzon> From my experimenting it seems that select is non-blocking, but because of that in my code it never returns the value sent from the channel because it runs before the channel sends
dwdv has joined #crystal-lang
ht_ has quit [Remote host closed the connection]
<FromGitter>
<watzon> Ok it's been hours of experimenting and I'm still no closer. I need someone like @asterite, @bew, or @straight-shoota who is more familiar with concurrency to help me out if you could. Basically this is the code giving me issues: ⏎ ⏎ ``` code paste, see link``` ⏎ ⏎ The full code for the relevant areas can be found here (https://github.com/watzon/proton/blob/master/src/proton/client.cr) and here
<FromGitter>
<watzon> I'll post this to the forum too, for posterity
dwdv has quit [Ping timeout: 240 seconds]
dwdv has joined #crystal-lang
yukai has quit [Ping timeout: 250 seconds]
ua has quit [*.net *.split]
bazaar has quit [*.net *.split]
bazaar has joined #crystal-lang
ua has joined #crystal-lang
HumanG33k has joined #crystal-lang
flaviodesousa has joined #crystal-lang
<FromGitter>
<j8r> How can I get information like with `openssl s_client -connect crystal-lang.org:443`?
<FromGitter>
<j8r> I would like to check the fingerprint of a TLS certificate
<FromGitter>
<j8r> I have a self-signed certificate on a local network, and don't want to manage a whole PKI infrastructure
<FromGitter>
<j8r> No `OpenSSL::SSL::Socket#cyphersuite` :(
<FromGitter>
<j8r> Anyway, perhaps someone has an other idea?
Vexatoast has quit [Quit: ZNC Quit]
Vexatos has joined #crystal-lang
hightower3 has joined #crystal-lang
<repo>
i noticed that there's no intersection method for ranges
<repo>
that'd be pretty handy
gangstacat has quit [Ping timeout: 276 seconds]
<FromGitter>
<yxhuvud> Hmm. I suppose both `&` and `|` could make sense on ranges.
<FromGitter>
<yxhuvud> It is not obvious how to represent an empty range though.
<FromGitter>
<Blacksmoke16> @watzon iirc its not documented much because it was/is changing a bit due to MT
<FromGitter>
<straight-shoota> > It is not obvious how to represent an empty range though. ⏎ ⏎ `0...0` is empty.
<FromGitter>
<straight-shoota> `range | range` though is impossible if the ranges don't overlap.
Human_G33k has joined #crystal-lang
HumanG33k has quit [Ping timeout: 240 seconds]
context has quit [Quit: quit]
<FromGitter>
<eliasjpr> Is there a way to emulate Scala ADTs in Crystal?
<FromGitter>
<tenebrousedge> for what purpose? probably the answer is no
<FromGitter>
<eliasjpr> for the purpose to have better type safe code and still extendable
<FromGitter>
<eliasjpr> with no dups
<Xeago>
repo: I'm trying to use crystal :> at the point now where gdb starts in vscode but it doesn't actually start the program.
<Xeago>
Also still lacking any form of intelligence when it comes to completion or type system in vscode.
<Xeago>
Running gdb bin/x directly works just fine.
<Xeago>
The debug console in vscode lists one thread starting, but no second thread as it does when running separately.
<FromGitter>
<tenebrousedge> did you check the crystal language extension settings?
gangstacat has joined #crystal-lang
<Xeago>
Is the default not to be useful in .cr files?
<FromGitter>
<tenebrousedge> probably that depends on how you define "useful", but completion, hover, and implementation search aren't enabled by default
<Xeago>
What is your definition of useful things you expect after installing the extension without further configuration?
hightower3 has quit [Ping timeout: 240 seconds]
<FromGitter>
<tenebrousedge> I usually use spacevim when I write Crystal, so I have no expectations about VSCode
<Xeago>
Understood.
<FromGitter>
<tenebrousedge> I would consider not enabling these features by default to be reasonable if they resulted in high CPU load / slow editor response
Human_G33k has quit [Quit: Leaving]
HumanG33k has joined #crystal-lang
foxxx0 has joined #crystal-lang
<foxxx0>
hey, are there some examples available regarding the parallel() macro to make it scale according to ENV["CRYSTAL_WORKERS"]?
<foxxx0>
i'd like to split the work to a varying amount of threads
<FromGitter>
<Blacksmoke16> would need to enable MT preview ofc
<foxxx0>
yes, sure
<FromGitter>
<Blacksmoke16> otherwise it should just work
<foxxx0>
it already works and i'm able to reach factor 2 speedup with 2.5-3.0 factor cpu load
<foxxx0>
yes, but i'm unable to understand how i would write the code to generically/dynamically scale according to the env var
<FromGitter>
<Blacksmoke16> not sure how its directly implemented, i.e. if changing the env var at runtime would scale the workers, or if its set when the program gets first executed
<FromGitter>
<Blacksmoke16> @bcardiff or @waj would prob know
<foxxx0>
changing the env var at runtime is not what i mean
<foxxx0>
as you can see i hardcoded the distribution to 4 parallel parts in lines 25 to 30
<foxxx0>
in order to change that according to the max workers setting at startup i would need a different approach, and that is what i'm looking for
<FromGitter>
<Blacksmoke16> oh, yea i dont think thats how it works
<FromGitter>
<Blacksmoke16> the parallel macro just spawns n number of fibers and waits for them to finish
<FromGitter>
<Blacksmoke16> the env var determines how many of those fibers can be executed in parallel
<FromGitter>
<Blacksmoke16> if that makes sense?
<foxxx0>
yes, but when i now want to set the number of fibers according to the env var so that they all can run concurrently, how would i achieve that?
<foxxx0>
i don't want to split the work into stupidly many fibers, i want to match the amount of workers
<FromGitter>
<Blacksmoke16> the default worker count is 4, so it should already do that assuming you build with `preview_mt`
<FromGitter>
<Blacksmoke16> so you should just spawn 4 fibers
<foxxx0>
but that wouldn't scale beyond 4 workers then?!
<FromGitter>
<Blacksmoke16> you can spawn more fibers than workers
<FromGitter>
<Blacksmoke16> just not all of them will be executed in parallel, only how many workers you have
<foxxx0>
yes, i got that
<foxxx0>
but i do not want to hardcode the amount of fibers
<FromGitter>
<Blacksmoke16> oh so all you want to do is spawn the same amount of fibers that there are workers?
<FromGitter>
<Blacksmoke16> in short?
<foxxx0>
yes!
<FromGitter>
<Blacksmoke16> couldnt you just do like
<foxxx0>
well i have an array of integers that i want to filter through. in order to parallelize that, i currently split it into 4 parts and hardcoded 4 fibers with parallel() where each fiber gets their slice of the original array
<foxxx0>
i don't see how i would do that with the snippet you posted
<foxxx0>
the fibers are not self-sufficient, i need scatter-gather or similar, i actually need to join the results of the fibers in the end
<repo>
i think you should try implementing a thread pool
<FromGitter>
<kinxer> @tenebrousedge Looking at the source (`Array#sort!`=> `Slice#sort!`), it looks like it uses Introsort (https://en.wikipedia.org/wiki/Introsort), which doesn't have better time complexity best-case than worst-case.
<FromGitter>
<kinxer> It would probably be faster to use Insertion Sort or a binary tree for this particular problem, but Introsort makes sense as an all-around choice for the standard library.
<FromGitter>
<tenebrousedge> hmmmm
<foxxx0>
are my comments regarding the fibers and multithreading correct? i want to make sure that i properly understand the terminology: https://paste.foxxx0.de/Q0s9/cr#n23
duane has joined #crystal-lang
<FromGitter>
<bcardiff> @foxxx0 one important thing that make things work in parallel is that if you do N spawn consecutively they are very likely to be allocated in different worker threads. But there is no API currently to ensure that. Since the current implementation of the scheduler is round-robin to select the worker thread then you get some balance. There is also no public API to know the number of workers, hence reading the
<FromGitter>
... env vars is needed. The whole idea is to make you think of fibers and not in threads. I expect some API improvements to express requirements for balancing the load but there is no draft yet. (I’m in an airport gate, so low connectivity)
<FromGitter>
<bcardiff> (N <= worker threads)
<foxxx0>
thanks, i just like to play around with it, even though it is experimental. using puzzles like advent of code is perfectly suited to try it out and stumble upon possible issues and limitations
<foxxx0>
although i'm seeing a lot of gc warnings: "GC Warning: Repeated allocation of very large block (appr. size 1576960): May lead to memory leak and poor performance"
<foxxx0>
and i head to export some env vars to make it scale: export GC_INITIAL_HEAP_SIZE=4G ; export GC_MAXIMUM_HEAP_SIZE=24G ; export MAXHINCR=32768 ; export MAX_HEAP_SECTS=16384 ;
<FromGitter>
<bcardiff> Definitely keep playing :-) The repeated allocations might be an array created in a tight loop. If you are sure the amount of memory is no a problem for short running apps you can even disable the GC. But is better if the intermediate allocations are reduced usually.
<foxxx0>
so i should do the splitting into separate arrays before spawning the workers and then just point them to their individual array upon spawning?
<foxxx0>
i could also try just passing them iterators
<foxxx0>
that is probably way more efficient but didn't perform as well for smaller workloads
<FromGitter>
<Blacksmoke16> what are `req.posts_list`? like `[1,2,3]` or?
<FromGitter>
<rishavs> Array(String)
<FromGitter>
<rishavs> array of uuids actually
<FromGitter>
<Blacksmoke16> dont have any thoughts off the top of my head, not used to doing raw queries like this
<FromGitter>
<rishavs> Thanks for helping anyway. This is an old thing with postgres; https://github.com/brianc/node-postgres/issues/623 I had thought maybe there would be an elegant solution to it by now
return0e_ has quit [Ping timeout: 240 seconds]
return0e has joined #crystal-lang
<dwdv>
What's the purpose of the UNDERSCORE type decl here? `def puts(*objects : _)`
<FromGitter>
<Blacksmoke16> accepts any type
<FromGitter>
<Blacksmoke16> like
<FromGitter>
<Blacksmoke16> `def foo(val )` is the same as `def foo(val : _)`
<FromGitter>
<Blacksmoke16> but IMO the latter is better as it shows you *want* it to be able to accept any type, vs there just not being one
<dwdv>
Explicitness, okay. Would you regard it as a code smell outside of the stdlib?
<FromGitter>
<Blacksmoke16> not really, is a language feature
<FromGitter>
<Blacksmoke16> i mean its a type restriction in of itself
<FromGitter>
<Blacksmoke16> just a more explicit way of saying "any type"
<FromGitter>
<tenebrousedge> `foo(val)` is totally normal
<FromGitter>
<tenebrousedge> `foo(val : _)` is equivalent
<FromGitter>
<Blacksmoke16> the 2nd is better imo
<dwdv>
And that would pass your code review when the new guy/gal adds these declarations in all new code?
<FromGitter>
<tenebrousedge> whether or not to use explicit type declarations is something I would expect to be part of a company-wide style guide
<FromGitter>
<tenebrousedge> but the Crystal book has something to say about this
<FromGitter>
<Blacksmoke16> > *<dwdv>* And that would pass your code review when the new guy/gal adds these declarations in all new code? ⏎ ⏎ assuming there isn't a more strict type that that method should accept then yes
<FromGitter>
<tenebrousedge> > So, in general, it's preferable not to specify type restrictions and almost only use them to define different method overloads. This results in more generic, reusable code. For example, if we define a class that has a + method but isn't a Number, we can use the add method that doesn't have type restrictions, but we can't use the add method that has restrictions. ⏎
<FromGitter>
<Blacksmoke16> using the `_` imo shows that that method should take any type, versus it just missing a type restriction
<FromGitter>
<Blacksmoke16> e.x. so someone comes along sees a method without a restriction and tries to type it to something, when it shouldnt be
<FromGitter>
<asterite> I think type restrictions are not bad and should be preferred. Maybe that section in the book should be updated. The example is pretty contrived anyways. But I guess in the end, unless we enforce it, it's up to the users
<FromGitter>
<tenebrousedge> why should they be preferred?
<FromGitter>
<Blacksmoke16> i like them because they help with the docs :p
<FromGitter>
<Blacksmoke16> and make compile errors more obvious
hightower2 has joined #crystal-lang
<hightower2>
Is Ary's recent talk recorded somewhere?
<FromGitter>
<tenebrousedge> isn't that scheduled for later today?
<hightower2>
oh cool, thought it already happened
<FromGitter>
<watzon> Ok @Blacksmoke16 @asterite this is basically what I've been trying to accomplish https://carc.in/#/r/84xg
<FromGitter>
<watzon> I know this code obviously won't work, but hopefully it will illustrate what I'm trying to accomplish
<FromGitter>
<watzon> Idk if I'm just thinking about things wrong or what
<hightower2>
watzon just based on a cursory look I didn't quite understand what each loop is doing, but is the issue maybe that you're not using Fiber.yield to explicitly pass control to whatever other fiber needs it?
<hightower2>
also I don't see where you are calling wait_for... this might affect what the answer/solution is
ur5us has joined #crystal-lang
sagax has quit [Read error: Connection reset by peer]
ur5us_ has joined #crystal-lang
ur5us has quit [Ping timeout: 245 seconds]
<Stephie>
i cant believe progress is finally being made on `assert`#
<Stephie>
it's a dream come true :)
<Stephie>
it's been, what, 3 years?
<FromGitter>
<watzon> What kind of progress are we talking about?
return0e has quit [Ping timeout: 250 seconds]
return0e has joined #crystal-lang
ur5us_ has quit [Read error: Connection reset by peer]
<repo>
dwdv: 0.035620 0.019525 0.055145 ( 0.040522) for test input: 125732329-579381427¬
<repo>
you can give it a go with your solution :)
<repo>
(running on a thinpad x270 i5)
ht_ has quit [Remote host closed the connection]
<dwdv>
Around 10x faster than my naive 8 line solution.
<repo>
did you try the test input i posted? :)
<repo>
this: 125732329-579381427
<repo>
results should be 14361 and 10394 respectively
<dwdv>
Alright, that takes forever. Nice.
<repo>
yeah there are some tricks as usually with AoC
ur5us__ has joined #crystal-lang
<repo>
mind you, i would never write that kind of code for any serious project, but it's fun to tinker with the solution and try to optimize (often in exchange for readability)
<dwdv>
Did you also write a full-fledged transpiler for the elfcode last year?
<repo>
no last year i didn't finish. i had a project deadline for start of january
<repo>
0.093397 0.007154 0.100551 ( 0.056140) btw for the same input with my not-as-horrible solution
<repo>
also not needing the pw length at compile-time
<FromGitter>
<tenebrousedge> I think that was a bug that's fixed in the next version
<dwdv>
repo: Try beating that: Moving my dlang solution into a seperate function without any other modification and forcing a compile-time evaluation via `enum t = foo("128392-643281");` solves the whole damn thing up front and bakes the result into the executable without any macro magic. Just takes a few mins to compile since it's interpreting it. :)
<Flipez>
Ah yes, I does work with an older version
<FromGitter>
<jgillich> I'm writing a Redisgraph client and for its query builder I'd like to have a function for each Redisgraph command. Unfortunately some commands are reserved keywords in Crystal (return, with, union). What's the best way around that? I thought of doing _return or RETURN, both are a bit ugly. Any ideas?
<FromGitter>
<tenebrousedge> symbols are probably a thing there
<FromGitter>
<jgillich> Do you have an example? I can't use symbols to define methods, or can I?
<FromGitter>
<tenebrousedge> no, and you can't use reserved keywords either
<FromGitter>
<tenebrousedge> I mean, you can use symbols to define methods, using macros
<FromGitter>
<Blacksmoke16> how would you call it tho
<FromGitter>
<Blacksmoke16> might be fine assuming its not a top level method
<FromGitter>
<tenebrousedge> I mean, it does let you do that