<SeanTAllen>
it takes a while to get a handle on it
<SeanTAllen>
i watched it many times
<SeanTAllen>
and kept asking sylvan questions
<SeanTAllen>
it became easier and easier as i started writing generic code
<obadz>
hehe
<obadz>
makes me feel better
<obadz>
when an actor needs more memory, what happens? I vaguely recall one of the videos I'd watched mentionned mmap?
<SeanTAllen>
wow ok obadz, i think that `fun ref foo` might be missing from the tutorial.
<SeanTAllen>
bad bad us.
<obadz>
I thought it wasn't totally new to me so I must have read about it somewhere
<obadz>
unless it was mentionned in a video
<SeanTAllen>
obadz: each actor has its own heap, that is assigned from a pool, so... if it needs more memory, it goes and gets more until there is no more memory
<SeanTAllen>
if its mentioned somewhere obadz, im not finding it
<obadz>
yeah I didn't find it either
<obadz>
probably a vague recollection from a video then
<SeanTAllen>
could you open an issue for that against the tutorial obadz ?
<SeanTAllen>
definitely should be there
<SeanTAllen>
and if you feel like taking it on, PRs are welcome.
<obadz>
when the heap needs to get extended, what happens if some other actor is already located right after the one that's trying to extend?
<SeanTAllen>
memory for the actor isn't guaranteed to be contigous
<SeanTAllen>
unless its a machine type or you use the "embed" keyword
<obadz>
Will open the issue but I feel I'm on very shaky grounds to write stuff about this :) Maybe if nobody's closed it by the time I get more comfortable
<SeanTAllen>
sounds good
<obadz>
so if you try to allocate a 200 bytes array in the actor, and only 100 bytes are available, you leave the 100 bytes unused and get more memory some place that can fit the whole array?
<obadz>
fun ref ⇐ what's that called? methods capabilities constrained by viewpoint on receiver?
<SeanTAllen>
if you try to allocate 200, if will go try and get a chunk from the pool
<SeanTAllen>
if the pool doesnt have enough for that
<SeanTAllen>
it will allocate another chunk and get you the memory from that
<obadz>
pool meaning, chunks of memories that already belong to that actor?
<SeanTAllen>
if the actor is already scheduled, then it will stay scheduled on the scheduler it is on
<SeanTAllen>
if a scheduler doesnt have any work
<obadz>
SeanTAllen: does that mean that a single actor could end up with chunks in pools that belong to different scheduler threads?
<SeanTAllen>
it will try to steal an actor to run from another scheduler
<obadz>
s/end up with/end up owning/
<SeanTAllen>
obadz: i only dug into that once several months back, so i would need to refresh my memory
<obadz>
Is there a documentation somewhere?
<SeanTAllen>
the code in libponyrt/mem/
<SeanTAllen>
beyond that, no
<obadz>
ok, thanks :)
<obadz>
I'll try to understand generics before memory layout :)
<SeanTAllen>
in general
<SeanTAllen>
allocate all the memory you think you will need ahead of time
<SeanTAllen>
to avoid additional allocations
Guest45611 has quit [Quit: ...]
<SeanTAllen>
the last time i really dug into that was back in September
<SeanTAllen>
to find an issue related to fragmentation
<SeanTAllen>
back then i understood it quite well
<SeanTAllen>
but my memory has faded from lack of use
<obadz>
in one video Sylvan claims that fragmentation is quite limited in Pony
<SeanTAllen>
whereas, i have spent a ton of time on work stealing, scheduling etc so that is all very free in my memory
<SeanTAllen>
obadz: i would say that is a true statement
<obadz>
Cool, I'd really like to understand the GC paradigm at some point
<obadz>
The kind of questions I have are, if actor A alocates some value V, and sends it (iso) to some other actor B, and references to actor A no longer exist, does the heap of actor A stay alive just because B keeps a pointer to V ?
<SeanTAllen>
for day to day use, i know 80% of the gc and then need to refresh my memory on some detailed bits. the ORCA paper is fairly straightfoward
<SeanTAllen>
yes
<SeanTAllen>
actor A currently keeps it alove
<obadz>
isn't that a problem?
<SeanTAllen>
s/alove/alive/
<obadz>
if an entire heap is kept alive just because of a 1 value?
<obadz>
I guess actor A can GC itself and the heap would get reduced to that 1 value
<SeanTAllen>
there are situations where it could be a problem
<obadz>
maybe that's not such a problem though you keep the few bytes of actor overhead?
<SeanTAllen>
the other option would be to copy it from one actor to another
<SeanTAllen>
and that
<SeanTAllen>
would be a large perf hit
<obadz>
right, no free lunch
<obadz>
that's the erlang model I believe
<SeanTAllen>
one of the nice things with ref capabilities is you can do zero copy message passing
<SeanTAllen>
yes
<SeanTAllen>
erlang does a copy on send for anything that isnt a binary
<SeanTAllen>
You can certainly get yourself into trouble with the GC right now
<SeanTAllen>
Sylvan and I have discussed a variety of strategies to address
<SeanTAllen>
Hopefully we will start playing around with them soon
<SeanTAllen>
A definite anti-pattern would be to allocate memory in one actor, send it to another actor and have the other actor hang on to it for a long time. If you did that over and over and over
<SeanTAllen>
Like say
<SeanTAllen>
where you are listening on a socket and allocate memory for the incoming data then send to another actor
<obadz>
right
<SeanTAllen>
you'd have a high GC cost in the network actor
<obadz>
wouldn't that be the natural way to organize the code though?
<SeanTAllen>
one approach to lower that would be to do generational GC
<SeanTAllen>
you can manually avoid that issue by doing a copy of the memory in the receiving actor
<SeanTAllen>
which isnt ideal
<obadz>
this video is so dense it's taken me 2 hours to watch 17 minutes of it :)
<SeanTAllen>
it gets easier
<SeanTAllen>
it is far and away the most dense of all the VUG videos
<SeanTAllen>
back to the GC...
<SeanTAllen>
the issue with what i spoke of is that the GC is a mark and sweep
<SeanTAllen>
so the more of those bits of memory that were allocated for objects that are sent elsewhere, the more to mark and sweep
<SeanTAllen>
for a large chunk of use cases, its very fast and very efficient
<SeanTAllen>
for some other uses cases, we have work to do
* obadz
is fascinated by GC
<SeanTAllen>
the startup i work for, Sendence, is open sourcing a high performance steaming data analytics platform soon and we've contributed back alot of performance improvements, found some outstanding bugs etc in the runtime
<SeanTAllen>
this specific gc case is one that interests us greatly and i spend a decent amount of time discussing options with Sylvan
<SeanTAllen>
Another area that needs improvement
<SeanTAllen>
If you send from actor A to actor B
<SeanTAllen>
that is the best case for the ORCA gc algo
<SeanTAllen>
if you send A to B to C
<SeanTAllen>
or even more in a chain
<SeanTAllen>
you get more and more overhead
<SeanTAllen>
and performance is impacted
<SeanTAllen>
when you learn how ORCA works, that will make sense
<SeanTAllen>
that is also an area of high concern for us at Sendence and something we are discussing with Sylvan how to improve
<SeanTAllen>
If you decide to use Pony for a project, we will happily discuss your problem and if you might run into a "not optimal" problem that we are aware of and perhaps can help you design around or we can all work together to start fixing said problem.
<SeanTAllen>
Every tool, every language makes lots of tradeoffs. We will never pretend that Pony is perfect or suitable for all problems.
<obadz>
Understood, thanks. I'm mostly just a curious & interested dev with nothing specific in mind
<obadz>
I'll try to read the ORCA paper soon :)
<obadz>
For now I have to head to bed, thanks a lot for all the help!
<SeanTAllen>
you're welcome
<SeanTAllen>
if you need more help in the future and no one is around on IRC, feel free to hit up the mailing list or ping me via email
Matthias247 has quit [Read error: Connection reset by peer]