endformationage has quit [Ping timeout: 268 seconds]
endformationage has joined #ponylang
endformationage has quit [Quit: WeeChat 2.3]
erip has joined #ponylang
erip has quit [Ping timeout: 245 seconds]
_whitelogger has joined #ponylang
PrsPrsBK has quit [Quit: PrsPrsBK]
profetes has joined #ponylang
srenatus has joined #ponylang
PrsPrsBK has joined #ponylang
<profetes>
hi, I'm investigating GC issue further and I found https://github.com/ponylang/ponyc/pull/2800. And PR 2709 as well. There's const of 1000 of actors asked if they are blocked. This should converge fast for most cases, but in mine (50k+ actors in one unit of work) it may converge too slowly (up to 5seconds) In that time another hundreds of actors get created. So GC is not able to reach again to actors created in the beginning of
<profetes>
the program for a looong time. Potentially ever.
<profetes>
I presume there should be some advanced heuristics (like in JVM: generations) to reach older data more frequently. Or make this 1000 limit something dynamic, that ensures that all actors would be checked in some reasonable amount of time
<profetes>
I mean if there's 100k actors - limit should be increased to converge in time
<profetes>
and this may still not be enough, so maybe some heuristics for "emptiness" of disjoint graphs should be kept, so graphs mostly blocked would be checked more frequently, to let whole disjoint graph go: in my case - unit of work of LetterFilter
<profetes>
does that makes sense? Is this probable enough to continue investigation in that direction?
<profetes>
does that make sense? Is this probable enough to continue investigation in that direction?
<vaninwagen>
it does, at least to me. Best thing is you create a github issue on the ponyc repo. i'll add a "needs discussion during sync" message, so the right people gonna have a look at it.
<profetes>
okay, I'll summarize what I got here and submit and issue.
<vaninwagen>
yeah, i think that makes most sense. :)
Foaly has quit [Quit: Now 'mid shadows deep falls blessed sleep.]
so has quit [Ping timeout: 246 seconds]
so has joined #ponylang
srenatus has quit [Quit: Connection closed for inactivity]
profetes has joined #ponylang
Amun_Ra has quit [Ping timeout: 272 seconds]
Amun_Ra has joined #ponylang
erip has joined #ponylang
<SeanTAllen>
profetes: re question you asked in aturley's twitch. "cast" isn't a term i would ever use in association with Pony.
<SeanTAllen>
refcaps are compile time only, they don't exist at runtime, so it's not what i would call "casting", "casting" to me is a runtime operation
<profetes>
i could not better term fast enough. How to call this operation that happens when I do recover ... end?
<profetes>
*i could not find..
erip has quit [Remote host closed the connection]
<SeanTAllen>
not quite sure what you mean profetes
<SeanTAllen>
cover creates a space where you can do mutable things while only allowing only immutable things in, which in turn allows you to do anything you want with the result
<SeanTAllen>
so for example, mutate something in place then return it as a `val` because there's no way to a mutable reference to escape
<SeanTAllen>
which is different than binding something like String iso^ to a String val
<profetes>
sure, I tried to understand thing I do like: ```let x = recover val String end``` when I want to change refcap from default one
<profetes>
and now i know that is another thing
<profetes>
and it's the iso^ that allows you to get any(?) refcap you specify
<SeanTAllen>
yes
<SeanTAllen>
iso^ is very powerful
<profetes>
and I like it, because I can stop overusing recover ... end, which also works for iso^
<SeanTAllen>
"no references to this currently exist, do whatever you want. its party time!"
<SeanTAllen>
and you never want to see iso!
<profetes>
indeed
<SeanTAllen>
"o o, so you had an isolated reference and you tried to alias it! you are a bad bad person. no biscuits for you!"
<profetes>
;)
<profetes>
since you're here - I've been investigating this issue with GC or my code
<SeanTAllen>
i saw
<profetes>
and I've filed an issue, but I'm still working on that
<SeanTAllen>
its a very interesting theory
<SeanTAllen>
seems quite possible
<profetes>
probably not true, if I do @sleep(15) - GC does nothing
<profetes>
so it must be related to dangling references somewhere
<profetes>
that's my current theory
<profetes>
but I have no source of information currently, since GC messages are unavailable to me. Do you know how to get GC debug output? :)
<profetes>
I keep reviewing my code, every promise, but it's a needle, and haystack is brand new
<SeanTAllen>
have you tried playing with the gc flags?
<SeanTAllen>
--ponygcinitial
<SeanTAllen>
--ponygcfactor
<SeanTAllen>
set both to 1 and pony will aggresively gc, pretty much after every single behavior call
<SeanTAllen>
its slow but can be useful in times like this
<profetes>
okay, that's perfect new knowledge, I'll try it now
<profetes>
do I need debug build?
<SeanTAllen>
you do not
<profetes>
no change, GC is still not freeing it.
<profetes>
I've reduced number of actors, still no change. Only Decoder get GCed. This means GC is working okay
<SeanTAllen>
well, it means, it is likely gc is working ok
<SeanTAllen>
one thing ive learned debugging is: "never say something is definitely working".
<SeanTAllen>
btw, how are you determining that gc isn't freeing it?
<profetes>
true. so to be precise: that experiment shows that most probable scenario I've suggested is most probably false
<profetes>
I've inserted @puts into finalize()
<profetes>
i mean fun _final()
<profetes>
it gets printed for some objects, but not this linked list or actor holding refenence to it
<SeanTAllen>
k
<profetes>
and then my program slows down, waits until swap is full and gets killed
<SeanTAllen>
sad sad times
<profetes>
Oh, if you knew :) It's AoC day 5 quiz. I started it back in December. First it took me ages to make actor cooperation stable via FSM, and then to return results from that. And now this.
<SeanTAllen>
:(
<profetes>
Always learning. I'm gonna be so satisfied when this is solved :)
<SeanTAllen>
well dump_views is the cycle detector
<SeanTAllen>
that im not aware of
<profetes>
okay. Do you know if GCing objects works the same way as GCing actors? I mean - it's the same logic/code?
<SeanTAllen>
well an actor is an object except you have the "might receive messages" aspect of "can this be gc'd". are you asking if the cycle detector is applied to objects?
<profetes>
yes
<SeanTAllen>
cycle detector is actor only
<profetes>
Thank you. I just observed, that only objects were freed, no actors. Might be a coincidence, I'm checking it further.
<profetes>
Well, it's GC for you, LetterFilter is "work unit". Some were freed
<profetes>
LetterFilter has some 50k actors inside. And they are run sequentially. First I tried to run all 26 pairs of letters simultaneously. BIG NOPE
<profetes>
I presume you're like 16GB of RAM?
<SeanTAllen>
i am
<profetes>
so apparently you can do 8*50k actors before crash. Nice!
<profetes>
On the other hand - I created NoopActor, it just prints a string. I put 3 of those in beginning/middle/end of program. None of them was freed by GC by the end of program + sleep(25)
erip has joined #ponylang
<SeanTAllen>
how does Part2Runner ever exit?
<profetes>
it sure do, limit number of letters in Range in _queue to a few
<profetes>
it sure does*
<profetes>
part2runner is part1 run 26 times sequentially
<profetes>
not an easy task in actor world
<profetes>
oh, and back to my NoopActor. Despite printing single message it does not get freed. But 2 Units (elements of double lined list) got freed by the end of a program.
<SeanTAllen>
how are you determining that?
<profetes>
by _final() in each actor doing @puts
<profetes>
currently in repo it's commented out for Unit
<profetes>
then I truncated the input in file to 450 letters, so only 450 actors are created in each list. Program does finishes correctly for me then.
<profetes>
I meant - did you tune anything with program/GC? With 3GB of RAM I get killed on letter W, way before S
<SeanTAllen>
no
<SeanTAllen>
can you try that change above and let me know what happens?
<profetes>
sure,
<SeanTAllen>
i cant get an answer for S
<SeanTAllen>
it ends up dying
<SeanTAllen>
what is doing for these letters?
<profetes>
increase the limit from 200 to something more
<profetes>
it's removing them from input text, a then "reacting"
<SeanTAllen>
reacting?
powerbit has joined #ponylang
<profetes>
looks like currently dying when doing M
<profetes>
not dead yet, I'm waiting
<profetes>
looks like stuck on M, swapping heavily, but not dead yet
<SeanTAllen>
what is this "reduced"?
<profetes>
"polymer"
<SeanTAllen>
i dont know what that means
<SeanTAllen>
in LineReaderWithFilter
<profetes>
like in those games, where you reduce the same symbols by clicking on them
<profetes>
from the input it reduces letters a+A, b+B, c+C, and so on. After reduction - new pair gets created, of neighbors of letters that were reduced=removed
<SeanTAllen>
i dont really understand
<profetes>
totally abstract quiz
<profetes>
ie. from dabAcCaCBAcCcaDA you get dabCBAcaDA
<profetes>
and in part2 you remove one letter from whole input, and then react the rest. And by this you find the shortest "polymer".
<profetes>
M died. I'll remove gcflags from program and retry
<SeanTAllen>
i can see tons and tons of memory will be used by this
<profetes>
True. But individual parts get computed pretty fast. And each part is independent from another. So I was hoping that GC will remove unused parts(LetterFilters), i just need one integer out of that, and I send it via Promise
<SeanTAllen>
so was your thinking that each LetterFilter would be gc'd after it was done?
<profetes>
yes
<profetes>
since no references are hold to it, and no messages are sent anymore
<profetes>
first I tries to run all letter-variants together. But it dies immediately. So I tried sequential approach. I know problem is artificial, synthetical and nasty. That's why I picked it for AoC, to test stuff on edge cases.
<SeanTAllen>
ummm
<SeanTAllen>
i dont see anything about no more references so far
<SeanTAllen>
when exactly is reacting done?
<SeanTAllen>
im done in Unit.report at this point and i see nothing that would allow any LetterFilter to be gc'd yet
<SeanTAllen>
the only thing i can see that is gc'able until its done is the Decoder
<profetes>
when all actorr=letter=units can no longer react with their neighbors. Then I pass Report iso into the list and if it comes on the other side it means there were no nodes reacting in whole list. otherwise reporting gets cancelled and tried later
<SeanTAllen>
there appears to be a long chain on of promises that manages to hold on to everything til its done
<SeanTAllen>
but its hard to follow
<SeanTAllen>
but what i see it doing, looks like what i would expect as best as i can follow what is happening
<profetes>
true, but they are all fulfilled when polymer is done
<profetes>
i know, that's why I asked a month ago how to return value in async way. The only way I know are the promises or some kind handmade pub/sub
<profetes>
so if I understand you correctly - holding a reference to a fulfilled promise prevents GCing?
<SeanTAllen>
im not sure what you mean by "they are all fulfilled when polymer is done"
<SeanTAllen>
i'm just following along and i continue to see something referencing things that reach back up to the letter filter
<profetes>
correct.
<SeanTAllen>
in _finish_and_report2
<profetes>
and then to Promises.join
<SeanTAllen>
why is result given a promise?
<profetes>
because i wanted eveerything to run at once, not sequentially
<profetes>
oh, the Result?
<SeanTAllen>
yes
<profetes>
because it may fail to collect the results from list (some units may still be reacting). And then I have to retry, if promise is rejected
<SeanTAllen>
if what promise is rejected?
<profetes>
I retry via ReactionWatcher - only when number of reactions reaches local 0
<profetes>
and then again - Result is passed
<profetes>
only one at a time
<SeanTAllen>
i dont really follow
<SeanTAllen>
sorry
<profetes>
(if you see better way of architecturing this - i'm all ears)
<SeanTAllen>
it seems like everything is being kept alive via promises until the final result can be computed
<SeanTAllen>
and so nothing would be gc'd
<SeanTAllen>
as best as i can figure, its doing what it should
<profetes>
okay, so how to pass result out?
<profetes>
so that letter filter can be freed?
<SeanTAllen>
i dont really understand that question
<profetes>
each LetterFilter produces and U32. I need to collect them and pick the minimum
<profetes>
*produces an U32
<profetes>
is there a language structure/pattern to do so, and let GC clean up, before all LetterFilters are done?
<SeanTAllen>
have you seen aturley's solution to day 5?