glenab has quit [Remote host closed the connection]
glenab has joined #crystal-lang
ponga has quit [Ping timeout: 250 seconds]
ddfreyne_ has quit [Ping timeout: 250 seconds]
hplar has quit [Ping timeout: 250 seconds]
NeverDie has quit [Ping timeout: 250 seconds]
iamstef has quit [Ping timeout: 250 seconds]
slash_nick has quit [Ping timeout: 250 seconds]
slash_nick has joined #crystal-lang
drizz has quit [Ping timeout: 256 seconds]
slash_nick has quit [Ping timeout: 250 seconds]
drizz has joined #crystal-lang
NeverDie_ has joined #crystal-lang
iamstef has joined #crystal-lang
ddfreyne_ has joined #crystal-lang
glenab has quit [Remote host closed the connection]
slash_nick has joined #crystal-lang
ponga has joined #crystal-lang
glenab has joined #crystal-lang
Philpax has joined #crystal-lang
hplar has joined #crystal-lang
glenab has quit [Remote host closed the connection]
glenab has joined #crystal-lang
glenab has quit [Remote host closed the connection]
<Philpax>
do structs have finalizers?
<dzv>
asterite: atomic ops are needed for lock free lists in scheduler, io and other places. what would be the best method to get them in crystal? llvm has atomrmw, compxchg and fences. if i have those i can implement most lock free data structures
glenab has joined #crystal-lang
glenab has quit [Ping timeout: 252 seconds]
glenab has joined #crystal-lang
ssvb has quit [Ping timeout: 268 seconds]
kulelu88 has quit [Quit: Leaving]
<dzv>
philpox: doesn't look like it
ssvb has joined #crystal-lang
<crystal-gh>
[crystal] technorama opened pull request #1620: Return nil or correct value in various methods. (master...f/leaky_returns) http://git.io/vcTWQ
tomchapin has joined #crystal-lang
glenab has quit [Remote host closed the connection]
glenab has joined #crystal-lang
glenab has quit [Remote host closed the connection]
<pawnbox>
I think old implementation of symbol was something like this.
<pawnbox>
in Ruby.
<jokke>
jhass: any ideas regarding my code snippet above?
<pawnbox>
Am I wrong?
<jokke>
yeah
<jhass>
jokke: no
<jokke>
pawnbox: ruby used to have memory leaks in some version > 2 because of symbols
<jokke>
.to_sym being the major problem
<jokke>
it's used excessively in rails for example
<BlaXpirit>
:<
<jokke>
iirc they weren't GC'ed
<pawnbox>
jokke: I just wanted to know how different is crystal's implementation of symbol than old implementation of symbol in ruby?
<BlaXpirit>
a good way to explain Crystal's symbols is... it's a global enum that automatically gets new members
<BlaXpirit>
i have no idea about ruby, all i know is current implementation isn't nearly as optimal
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
<jokke>
jhass: that was just stupidity on my account
<jokke>
jhass: i "overwrote" read_slice and write_slice instead of read and write in MaskIO
<jokke>
is_a?(MaskIO) is true anyway
<BlaXpirit>
we need an override specifier to def
<BlaXpirit>
that fails if there is no previous def
RX14 has quit [Ping timeout: 240 seconds]
RX14 has joined #crystal-lang
<jokke>
that'd be really useful
<jokke>
i have _a lot_ of data i need to write to and read from an IO. it won't fit into ram. i thought of using IO.pipe(true, true) and then creating two threads that write/read from it. How would i do this in a crystalline fashion?
<jokke>
i admit i still quite haven't understood fibers and such in crystal
<dzv>
that's not going to work well
<dzv>
not currently
<jokke>
dzv: why not?
<dzv>
the threads you create can't use fibers or channels
<dzv>
so they won't be able to pass data between each other easily
<jokke>
i'm not sure i follow...
<dzv>
if they do nothing but read from io where blocking = true then threads will probably work. just don't call anything that may use the scheduler or spawn
<jokke>
hm ok
<jokke>
dzv: so do i spawn the threads manually with Thread.new?
<dzv>
yes
<jokke>
ok
<dzv>
and remember, no data structures in crystal are thread safe
<jokke>
yeah
<dzv>
i'm working on getting threadsto work transparently
<dzv>
with fibers
pawnbox has quit [Read error: Connection reset by peer]
<dzv>
an east way to think of fibers in crystal is just like threads. spawn { } creates a fiber, Thread.new { } creates a thread.
<dzv>
easy
<dzv>
fibers are scheduled by crystal internally, threads are scheduled by the kernel
<dzv>
they both do the same thing on a single processor. switch tasks.
<jokke>
ok
<jokke>
hm but wait
<jokke>
Threads spawn pthreads, right?
<jokke>
those should do things on multiple processors
<dzv>
yes
<dzv>
eventually there will be multiple threads running fibers
<dzv>
and if i can get my PR's through like #1612 they will exist sooner
<jokke>
anyone else? some feedback would be very nice since those are big changes..
<jokke>
it's mostly waj asterite and some dude kumpelblase2
<jhass>
Papierkorb said he might look into better Websocket support too
<jokke>
i think for being able to implement RFC conform websocket support it's necessary to have classes for frames which encapsulate the parsing and validation
<jokke>
also with #1603 one can simply do sth like frame = HTTP::WebSocketTextFrame.new; ...; websocket.write_object(frame)
<jokke>
next step would be to implement a handler that uses the frame objects and reacts to control frames accordingly (pong for ping and close for close)
<dzv>
jokke: did you handle the case where mask key is 0? (it's invalid)
<jokke>
dzv: i didn't. thanks for pointing it out.
<dzv>
you should probably allow receiving mask of 0, but never send by default using 0. also allow 0 if set explicitly
<dzv>
to allow for non rfc compliant clients
<jokke>
dzv: so basically keep generating keys until i get one that isn't 0
<dzv>
yes
<jokke>
dzv: is using securerandom a bit over the top?
<dzv>
probably
<jokke>
it could drain entropy pretty quickly if sending lots of fragmented frames
<dzv>
it's sent in the clear
<jokke>
too bad rand hasn't got random_bytes
<dzv>
it's using libcrypto. i don't think that drains system entropy as rapidly
<dzv>
there's an internal pool used after seeding from /dev/urandom. not sure how often the reseeding happens, but i think never. it's been 10 years since i read the code though
<dzv>
but i have > 6 branches with interdependencies. i can't mentally keep track of what is where and what depends on which
<dzv>
i'm at my limit
<jokke>
yeah i feel you :D
<dzv>
those 6 branches are all for a single feature: thread support
<jokke>
it's awesome that you work on it though!
<jokke>
it'll be a killer feature!
<dzv>
starting to get pissed... task switching is distracting. especially when you wait for days or weeks before getting feedback
<dzv>
and i can't just cherry pick commits. i may need to refactor and almost certainly rebase before the pr's are accepted
<dzv>
which screws up every other branch that depends on it
<jokke>
yeah
<jokke>
i'm afraid too that my branches diverge too much
<jokke>
fortunately i only have one dependency so far
<jokke>
but i think 6 open pr :)
<jhass>
dzv: I told you that before, I feel you but you're complaining to the wrong person
<jhass>
I'm not doing the language design here, I can't make these decisions
<dzv>
i wouldn't consider #1612 language design unless you're speaking of how an argument is passed to Mutex. Thread::Attribute just allows setting/getting additional data on an existing class written by waj and asterite
<dzv>
i need to get the stack size of a thread to work with the gc
<jhass>
I think whether to make subclasses or not is very much touching language design
<jhass>
given these are the synchronization primitives
<dzv>
are you speaking of Mutex or Attributes?
<jhass>
Mutex
<dzv>
fine, i'll split the PR. will you accept attributes? that's what i need right now
<jhass>
I guess I can justify that, yes
<dzv>
and i thought mutex would be the easy one to get through
<jhass>
there's also the question as to whether to map the phtread recursive mutex at all or do something like Ruby's Monitor that keeps track of the lock state and skips relocking, which is slower but gives much less surprising behavior
<dzv>
how does a recursive mutex give suprising behavior?
<jhass>
you even mentioned it in its docs
<jhass>
you deadlock it very easily
<dzv>
monitor handles that?
<jhass>
yes, it simply wraps a regular mutex and counts the calls to lock
<jhass>
and only unlocks when that counter reaches 0
<jhass>
well, a thread local counter I guess
<dzv>
didn't realize monitor worked around that problem
<jhass>
so whether to have distinct classes so you know whether you got a recursive mutex or not and whether to map it at all or go for something slower but easier to use like Monitor (and whether to even just call that implementation Mutex::Recursive etc) is very much language design
<jhass>
even though we intend for the main concurrency primitives hide all that
<dzv>
i'll accept Monitor and Synchronized. I'm at the point where I need them for testing.
<jhass>
dzv: what happens if I were to call Thread::Attributes.new and pass them to another thread?
<dzv>
nothing harmful unless you set the stack
<dzv>
it's meant to be reused
<dzv>
and not necessary to keep around after thread creation
<jhass>
Oh I see it's just a value container
<jhass>
how about adding a constructor that takes the stack size?
<dzv>
there are other things it will be used for in the future, like setting scheduling policy. imagine a group of threads/fibers that are high priority (web requests) and group of background threads/fibers. pthread_attr let's you change the scheduling policy
<dzv>
and a few other things
<dzv>
right now i don't need to set the stack size, i need to get the stack size returned after pthread_attr_init is called
<jhass>
maybe a thread instance should carry one around and delegate methods to it?
<jhass>
or do you have to set stuff prior creating a thread?
<dzv>
everything prior. the values aren't used after creation
<dzv>
and it may be reused when creating multiple threads
<jhass>
so what do you think about passing everything in its constructor?
<jhass>
possibly with nil defaults and only set it when given
<dzv>
well... if you changed the stack itself (not the stack size) then you couldn't reuse to create multiple threads
<jhass>
I think mapping it immutable would give an easier to reason about interface
<dzv>
there may be other settings that need to change between calls but i don't remember any off the top of my head
<dzv>
part of this is to work with the gc. i THINK i can make this work if i can get the stack size (there's a workaround) but yesterday i thought the only way was to mmap a new stack and set it explicitly
<jhass>
See, if you start to think about it affects the API to create threads greatly
<dzv>
honestly i think attr should not be exposed to the user. Thread.new should take arguments. that's the plan and that's why it's not exposed anywhere except as an arg to LibPThread.create
<jhass>
I agree
<jhass>
but there's not even a # :nodoc: tag on it
<dzv>
i have a PR that's not ready but is wating on attr support
<dzv>
ok, oversight
<jhass>
and an easy to reason about coherent change would be adding that user facing API
NeverDie has joined #crystal-lang
<jhass>
then including the backbone needed for it
<jhass>
if I merge this now I add code that's essentially unreachable to the regular user in the hope a sensible API will be added later
<dzv>
i have a pr i'm working on where i need the ptherad_attr to get or set the stack inside of Thread.new
<jhass>
whether it can even support the API that's then actually wanted, idk
<dzv>
there are no thread api changes in my next pr. it's just to make the gc work correctly with threads and fibers
<dzv>
that's why i need the attr
<jhass>
but do you see why it's hard for me to accept this? I'm not in your head
<jhass>
and I mean for the stuff you describe using the LibPThread functions directly in Thread.new would probably be fine
<dzv>
ok. asterite wants small pr's. i can make a mega pr that makes threads work everywheer that adds synchronized, monitors, atomic ops, fences, changes the scheduler, adds thread groups or i can try to break it up in to small pieces but no one can see the big picture
<jhass>
what you're doing is crushing it into parts
<dzv>
what do you suggest?
<jhass>
what would make it easy to accept your PRs is extracting in itself coherent smaller changesets
<dzv>
the gc needs to work with fibers? does pthread_attribute go there?
<jhass>
not "thing I need for this internal thing which will provide this internal thing which will provide this user facing thing"
<jhass>
for the current PR a coherent change would be adding the user facing API of creating a thread with a given stack or whatever the goal is
<jhass>
This is a lingering bone that may or may not support the API doing that
<dzv>
there is no user facing change at all
<dzv>
so far this is all internal
<jhass>
maybe not end user of the language facing, but there are more stakeholders than that
<dzv>
at least as far as the gc is concerned
<jhass>
user facing in the sense of public API from one component to a (hypothetical) other component of the system
<dzv>
pthread_attr is an internal component used by 1 other component
<dzv>
nothing public
<jhass>
I can only repeat myself, there's no motivation for that changeset
<jhass>
small PRs doesn't mean lingering parts or few locs
<dzv>
large pr's sit for weeks or months and i can't make any progress at all
<dzv>
so there's a bigger problem here
<jhass>
the problem is extracting the proper small pictures out of the big one
<jhass>
you plainly raster the problem and cut out arbitrary pieces, sorry
<jhass>
providing us the pieces of a puzzle
<dzv>
Process.spawn was a single small issue completely extracted but it still sat around for WAY too long before a decision was made or even feedback on what would be accepted
<dzv>
then i was told to make thing smaller
<dzv>
ok
<jhass>
dzv: you need to try to get into the head of a person reviewing this
<jhass>
you're not attempting that at all
<jhass>
you have your big picture and master plan and expect us to implicitly fit stuff into it
<dzv>
i am, it's just not working. if it "big" it sits for weeks regardless of it's a strictly confined to a single issue
<dzv>
even if it can't be cut any smaller without making it a puzzle
pawnbox_ has joined #crystal-lang
<jhass>
as it stands I see no motivation for the change there's no API for other components to use it
<jhass>
I'm not even asking for usage of it
<jhass>
I'm asking for a sensible interface to it
<dzv>
Thread.new will used it. Attribute.new then get or set the stack size and address
<dzv>
you can see where it will go in f/thread_safe_fibers
<dzv>
it replaces the platform specific stuff
<dzv>
and if you want me to include it there fine. but then i expect that pr to sit around for a month or so before decisions are made
<jhass>
if there's zero interface for other components then I do require usage
<jhass>
as it stands I would add a blob of currently useless code, sorry
<jhass>
and big stuff does take it's time
<jhass>
if you want to reduce time wasted you have to discuss the purse end user design beforehand
<jhass>
if you have an agreed user facing interface you can argue internal changes with it
<jhass>
and sometimes you do need a big PR
<jhass>
you were asked to send smaller PRs since you were often mixing multiple pictures into one
<jhass>
but sending puzzles in, it's not making it go any faster
<dzv>
jhass: you're punting to asterite or waj. you said i should try to get in their heads and look at it from their point of view. i am. i see 2 busy guys working on other things with their own interests. i make a big pr it sits for weeks or longer unless pressu. i try smaller it gets ignored
<dzv>
i ask them for guidance on irc sometimes i get an answer, half the time not
<jhass>
they're using bouncers, at least for asterite you can see his away status to see whether he's connected or not
<dzv>
i open pr's sometimes they answer, sometimes never: #1284
<dzv>
i ask in the channel, unless it's a simple question i generally don't get an answer
<dzv>
i have plenty of away messages from asterite
<dzv>
here's 1 i asked before opening a PR to remove the 2nd spawn: 16:49:23 dzv | can i remove the 2nd spawn and make spawn usage consistent?
<dzv>
never answered
<jhass>
there's nothing I can do, I asked them before to pay more attention to the PRs
<dzv>
so i created the PR
<jhass>
I can't just overtake the language either
rossjones has left #crystal-lang ["A+"]
<jhass>
I don't know what you expect from complaining towards me
<dzv>
which had issues and i proposed solutions
<dzv>
so i opened a 2nd PR
<jhass>
I warned them that they'll drive away contributors if they don't prioritize review
<dzv>
you? this isn't for you. i hope they read this
trapped has quit [Read error: Connection reset by peer]
<jhass>
maybe they need to experience it happen themselves, I'd be sad but it's the way it is sometimes
<jhass>
the alternative would be for you to finish your big picture and use it and we pull it into a branch into the main repo or something and leave it to them to change it to something they would want to merge
<jhass>
we sometimes do that over at diaspora for the really big stuff
<jhass>
have a WIP branch that other (collab or not) can work on
<jhass>
it works good if somebody submits something that's 70-90% there
<dzv>
ok, well for testing there's a lot of temporary stuff i would need to commit and then possibly back out. using Synchronized everywhere until lock free data structures are available
<dzv>
would something like that with lots changes be accepted ever?
<jhass>
I don't know
<dzv>
that's where i can't get in their heads. i have no idea what they will accept or why not
<jhass>
probably not as is
<dzv>
so that's what the attribute was. an attempt to try something self contained and small with no risk of breaking anything
<jhass>
but as said one way I see is having that big picture tap it somewhere onto the wall (a branch in the main repo for example) and then ask people to extract individual interfaces out of it
<dzv>
to see if MAYBE they would approve PR's faster that way and i could keep going
<jhass>
I tried to explain why there's no motivation to merge it
<dzv>
for you. i still don't know how asterite/waj will handle it
<jhass>
you replaced the problem of too many changes with changes that have no motivation
<dzv>
it seems like those are my only options
kulelu88 has joined #crystal-lang
<jhass>
btw having a fork for such big structural changes and upstream the parts when individual coherent interfaces stabilize to upstream review, the reintegrate the reviewed work into the fork and continue is a pretty normal process
<jhass>
see for example the kdbus PR to the kernel
willl has joined #crystal-lang
<jhass>
it allows all stakeholders to keep the review surface to a tangible size
<jhass>
but you rejected that path as too much work, which is of course a decision you're allowed to make
sooli has joined #crystal-lang
<sooli>
Hi there I have a little issue on my mac
<dzv>
i didn't reject it was too much work, i split this up in the hope that review would be faster in pieces as i thought asterite asked for
<jhass>
sooli: mmh, the wrapper should set LIBRARY_PATH and thus it should find it from /opt/crystal/embedded/lib
<kulelu88>
Does crystal have something like asyncio that python has?
<sooli>
I did install crystal via brew
<sooli>
and nothing in /opt/crystal
<dzv>
jhass: consider this an experiment in merge speed. i have few data points all of them tell me "too long"
<jhass>
kulelu88: stdlib IO is evented by default, if you spawn a coroutine and it blocks on IO another ready coroutine will be run instead
<jhass>
kulelu88: so simply spawn and do your regular "blocking" calls
<dzv>
kulelu88: all the async io is hidden behind fibers. io.read doesn't block the thread
<jhass>
sooli: do you have the path I mentioned?
mhib has joined #crystal-lang
<sooli>
no it’s empty
<dzv>
sooli: `which crystal`
<sooli>
but hold on I’m trying to check why it occurs on my script
<sooli>
on a basic one it works fine
<dzv>
jhass: how familar are you with the compiler? do you have any input on how i could get atomic increment working?
<sooli>
# which crystal -> /usr/local/bin/crystal
<sooli>
FYI, my script works fine on ubuntu … but not on my mac
<sooli>
it use amatista
<jhass>
dzv: I guess you would need to make the LLVM primitive available
<dzv>
i see the primitive definitions in builder. i just don't see how to expose those primitives in code
<sooli>
-lgc is about Garbage Collection, right ?
<sooli>
how can I be sure that everything is well installed on my mac ?
<dzv>
ls -l /usr/local/bin/crystal
<jhass>
dzv: looks like you have to add a Def to class whose last argument is a Primitive with a given name, then handle that name in codegen/primitives.cr
pawnbox_ has quit [Read error: Connection reset by peer]
<jhass>
most of these current definitions seem to be in types.cr
<RX14>
sooli, it looks to me that the crystalinstall failed somehow
greengriminal has joined #crystal-lang
<jhass>
dzv: some definitions are also in primitives.cr, for example pointer malloc
<jokke>
jhass: i'm trying to move the module IO::ByteFormat to ByteFormat and let it return only static arrays which then will be used by the to_io methods. however i get undefined constant ByteFormat when trying to run the specs. I added require "byte_format" to the prelude
<jokke>
any ideas what i might be doing wrong?
<sooli>
I plan to use a webservice in production modd
<sooli>
mode
<sooli>
I like living dangerously
<jokke>
:)
<jokke>
sooli: that's my goal too
<jokke>
i'll be writing a json api with crystal
sfcgeorge has joined #crystal-lang
<sooli>
for now my tool are : crystal of course, amatista & sqlite
<sfcgeorge>
jhass: I tried out your gtk bindings demo on OS X (after installing gtk+3 with Homebrew) and it works just fine :)
<jhass>
cool :D
sooli has quit [Quit: sooli]
<jhass>
jokke: sure you use bin/crystal ?
<jokke>
m(
<jokke>
of course not
mhib has quit [Ping timeout: 244 seconds]
pawnbox has quit [Ping timeout: 250 seconds]
<crystal-gh>
[crystal] jreinert opened pull request #1629: move byte_format to std/ for getting bytes of numbers without io (master...byte-format) http://git.io/vcI6d
n0xff has joined #crystal-lang
<jhass>
jokke: doesn't that hand out pointers to the stack now?
<jhass>
mmh, or I guess it does copies when returning the static array
<jokke>
yeah
<jhass>
I think the idea was to have it zero copy
<jhass>
well, or at least with least amount of copies possible
<jokke>
but the previous implementation did it in the same way
<jokke>
ah
<jhass>
it wrote it to the IO
<jokke>
i think i get the point
<jokke>
because of the return
<jhass>
yeah
Ven has joined #crystal-lang
<jokke>
what if i add a @[AlwaysInline]?
<jhass>
no idea
<jokke>
i _think_ that should do the trick
<jokke>
but i'd wait for asterite or waj to comment on that
<jhass>
yeah
<jokke>
i've come across so many points where i need the bytes of a integer now that it made sense to me to extract the code into std/
<BlaXpirit>
jokke, where is that code?
<BlaXpirit>
nevermind
sooli has joined #crystal-lang
Philpax has quit [Read error: Connection reset by peer]
Philpax has joined #crystal-lang
<crystal-gh>
[crystal] asterite pushed 4 new commits to master: http://git.io/vcIyk
<crystal-gh>
crystal/master 71645d8 Joakim Reinert: add support for polymorphic macro methods...
<crystal-gh>
crystal/master 9b574c0 Joakim Reinert: add specs for macro method return type checks
<rmosolgo>
that's a good way to find out: just run it! :)
pragmatism has joined #crystal-lang
<willl>
especially since the std lib is all just in crystal, and you can reopen classes in your own code, trying things like this is very nice
havenwood has joined #crystal-lang
Philpax has quit [Ping timeout: 268 seconds]
<crystal-gh>
[crystal] rmosolgo opened pull request #1630: refactor(String#includes?) use a single def for Char and String (master...string-includes-definition) http://git.io/vcLXl
<rmosolgo>
I guess we'll see :)
havenwood has quit [Ping timeout: 260 seconds]
havenwood has joined #crystal-lang
<crystal-gh>
[crystal] asterite pushed 4 new commits to master: http://git.io/vcL5J
<crystal-gh>
crystal/master 3e2ddda Ary Borenszweig: Fixed #1628: can't define a class named `Main`
<crystal-gh>
crystal/master ae44faf Ary Borenszweig: Codegen: don't define the symbol table if it's empty
<jokke>
it has a lot of refactoring potential though
alanwillms has quit [Quit: Saindo]
<jokke>
but the specs are quite reassuring for doing that
willl has joined #crystal-lang
<jokke>
so if you want to hack with me on this i can give you write access to my fork
<jokke>
two pairs of eyes is always better than one
<Papierkorb>
jokke: let me think about it tomorrow, quite late here already and can barely think right now
havenwood has quit [Ping timeout: 264 seconds]
<jokke>
Papierkorb: my goal would be to come out with a handler that would handle frames on the web socket, respond to control frames according to the RFC and pass data frames to the caller
<jokke>
Papierkorb: sure. i'm from germany too ;)
<jokke>
well, you know where you can find me :)
<Papierkorb>
jokke: Sure, will get back to you tomorrow then. I'm using the Autobahn TestSuite as solid test base.
<Papierkorb>
I have to get to know more about the possibilities, but I'm not a fan of rubys way of doing connections. Much more of a fan of event driven designs, which allow for concurrently handling multiple connections in a single thread (Yeah I'm a C++/Qt guy). I don't know yet though what crystals channels offer in this regard, no experience with that concept
rappo has quit [Quit: .]
leafybasil has joined #crystal-lang
rappo has joined #crystal-lang
havenwood has joined #crystal-lang
<asterite>
jokke: mmm... I don't like it. The current implementation is simple, shorter and efficient. This new implementation allocates a new class for every frame
<asterite>
I usually prefer to keep things simple if possible. I don't think end-users will have to deal with the individual frames, just the high-level API, so I don't think refactoring all of this for better OOP-ish look is worth it
havenwood has quit [Ping timeout: 250 seconds]
glenab_ has joined #crystal-lang
apt-get has quit [Remote host closed the connection]
trapped has quit [Read error: Connection reset by peer]