Matthias247 has quit [Read error: Connection reset by peer]
jemc has quit [Ping timeout: 258 seconds]
jmiven has quit [Ping timeout: 260 seconds]
kr1shnak has quit [Ping timeout: 264 seconds]
kempe has quit [Ping timeout: 260 seconds]
kempe has joined #ponylang
<tokenrove>
just curious; why was net/http/Client renamed to HTTPClient? it feels to me like it's nicer to `use http = "net/http"` and then have `http.Client` in your code rather than adding redundant package names. (i realize this pattern occurs elsewhere in the stdlib)
chemist69 has quit [Ping timeout: 260 seconds]
jemc has joined #ponylang
chemist69 has joined #ponylang
gmcabrita has quit [Quit: Connection closed for inactivity]
jemc has quit [Ping timeout: 260 seconds]
jemc has joined #ponylang
plietar has quit [Remote host closed the connection]
<killerswan>
linux builds are on Bintray as the README describes
<killerswan>
homebrew is on the way
<killerswan>
let us know if you have any trouble! <3
unbalancedparen has quit [Quit: WeeChat 1.7]
jemc has joined #ponylang
jemc has quit [Ping timeout: 240 seconds]
sigipa has joined #ponylang
killerswan has quit [Quit: killerswan]
amclain has quit [Quit: Leaving]
killerswan has joined #ponylang
plietar has quit [Remote host closed the connection]
plietar has joined #ponylang
plietar has quit [Ping timeout: 240 seconds]
plietar has joined #ponylang
plietar has quit [Remote host closed the connection]
plietar has joined #ponylang
plietar has quit [Ping timeout: 258 seconds]
<sigipa>
Hello All. Is anyone using pony on freebsd?
cousin_luigi has joined #ponylang
plietar has joined #ponylang
<sigipa>
Is anyone around?
gmcabrita has joined #ponylang
plietar has quit [Remote host closed the connection]
plietar has joined #ponylang
killerswan has quit [Quit: killerswan]
Praetonus has joined #ponylang
plietar has quit [Ping timeout: 258 seconds]
virtual_lark has joined #ponylang
plietar has joined #ponylang
<SeanTAllen>
sigipa: according to the community survey, yes but most support questions we get are for Linux, windows and OS X.
rosstuck has joined #ponylang
plietar has quit [Remote host closed the connection]
plietar has joined #ponylang
rosstuck has quit [Remote host closed the connection]
kr1shnak has joined #ponylang
plietar has quit [Ping timeout: 258 seconds]
<sigipa>
I don't know much about language internals and design, but I have (what may be a stupid) question about gc.
<sigipa>
It looks like the issue with the gc not collecting actors is still around. If I understand correctly, it has something to do with them not triggering gc due to insufficient garbage to collect.
mrkishi has joined #ponylang
<sigipa>
My question is.. why couldn't a couple of api calls be exposed to the gc to allow kind of a manual mode? You could have a "collect me" call at the end of each actor with another call to run the gc.
<sigipa>
The more I think about this, the more I would like the option in general. It kinf of fits the stuff I do. Would something like this be possible?
<sigipa>
I realize most folks probably wouldn't like this, but it could be a compile flag to enable it.
<sigipa>
Anyway, it would finally make pony viable for heavy workloads.
<SeanTAllen>
We use pony for heavy workloads so I think your "finally make Pony viable for heavy workloads" is a bit off base
plietar has joined #ponylang
<sigipa>
I'm just speaking from experience and from freebsd. My code does very few memory allocations. It will run until memory is exhausted and the OS will kill it. I meant no offense. I should have said "viable for me".
<SeanTAllen>
There's nothing FreeBSD specific about how the GC works.
<SeanTAllen>
If your code does very few memory allocations, I don't understand how memory is exhausted. That would require memory allocations. This is all too general to be a solveable problem at the moment.
<SeanTAllen>
There are command line flags to change how often the gc runs
<SeanTAllen>
See `ponyc --help`
<SeanTAllen>
--ponygcinitial
<SeanTAllen>
--ponygcfactor
<SeanTAllen>
There is already a way to force a gc after the current behavior finishes...
<sigipa>
The gc never collects actors. Once enough have been created it's over.
<SeanTAllen>
@pony_triggergc[None](this)
<SeanTAllen>
Your language is rather unclear
<SeanTAllen>
please define "collects actors"
<sigipa>
The problem I'm seeing is exactly what happens to the http server when you put it under load.
<SeanTAllen>
Sorry, I don't know what you mean
<SeanTAllen>
Are you referencing an open issue?
<sigipa>
I'm looking it up...
<SeanTAllen>
And can you define "never collects actors"?
<sigipa>
I assume that creating an actor does use some memory.
<SeanTAllen>
Sorry, I still dont know what you mean by "never collects actors"
<SeanTAllen>
Are you referring to quiescance and the lifecycle of an actor? or are you referring to gc being run on an actor's heap?
<sigipa>
issue 1417. There have been several other variations. I worked with Sylvan on it last year.
<sigipa>
As I mentioned previously, I'm not up on pony internals. I just know that the behavior is to run until memory is exhausted. If the gc were doing it's job, that should be possible.
<sigipa>
shouldn't
<SeanTAllen>
you are speaking of an edge case
<SeanTAllen>
but i dont have enough information to assist
<SeanTAllen>
you have pointed to one bug, which is good
<SeanTAllen>
but in terms of being able to suggest things that would help, that i can't do without knowing more about your specific use case
<SeanTAllen>
there are currently 3 ways you can impact on how the gc runs as i detailed earlier
<SeanTAllen>
--ponygcinitial
<SeanTAllen>
--ponygcfactor
<SeanTAllen>
@pony_triggergc[None](this)
<sigipa>
I was unaware of the gc calls. I think the next step is to try that. Thanks for the pointer.
<SeanTAllen>
Actor lifecycles are determined by reachability
<sigipa>
Is there a way to suspend the gc as well?
<SeanTAllen>
As long as an actor is:
<SeanTAllen>
referenced from another actor
<SeanTAllen>
has messages on its queue
<SeanTAllen>
is registered to receive asio events
<SeanTAllen>
then the actor will continue to exist
<SeanTAllen>
(thats the simple form)
<SeanTAllen>
if two actors reference one another but have empty queues and neither is registered to receive any events, then they will be reaped
<SeanTAllen>
there is no way to suspend the gc, you can just put off its running
<sigipa>
I tried using long running actors, but I could find a communication methos that was fast enough.
<SeanTAllen>
by adjusting the initial gc size
<SeanTAllen>
Sorry i dont know what you mean by "I tried using long running actors, but I could find a communication methos that was fast enough."
<SeanTAllen>
You are being very vague about the problem you are trying to solve so the best i can do is vague answers
<sigipa>
My project is a packet processing pipeline. It's sort of like DPDK.
<SeanTAllen>
If you create lots of short lived actors that all use some memory, and you hold on to references to them, then they won't get reaped because of the references and if they are below the gc trigger point, that memory won't be freed by gc
<sigipa>
Currenly it is written in c and uses lock free queues to communicate between pipeline nodes. I was trying to see if pony would work as an alternative to c.
<SeanTAllen>
So this is coming like A sends to B sends to C ?
<sigipa>
There are many branches, but basically yes.
<SeanTAllen>
So, memory arrives over a socket at A and is then sent to B and on to C?
<SeanTAllen>
or does a send to b and b does something and allocates new memory and sends to c?
<sigipa>
The queues are in shared memory. The memory is allocated in a big slab, so the nodes themselves do very little allocation.
<SeanTAllen>
That's the C version I assume
<SeanTAllen>
I'm asking about what you tried in Pony
<SeanTAllen>
I still dont understand "I couldn't find a communication methos that was fast enough."
<SeanTAllen>
I'm trying to understand what "wasn't fast enough"
<sigipa>
The queues are a ring of stacks. Packets are read of of the wire until one stack is full and then pushed down the line.
<SeanTAllen>
I've worked on writing high performance pony
<SeanTAllen>
I can tell you this
<SeanTAllen>
At this point in time, you need to understand some internals to do that
<sigipa>
I just tried to replicate the same basic structure.
<SeanTAllen>
Sending memory allocated in actor A via an object to actor B and then actor C
<SeanTAllen>
is currently an anti-pattern for how ORCA the gc algo works
<SeanTAllen>
its an issue that needs to be addressed
<SeanTAllen>
i dont know that you had a problem with that
<SeanTAllen>
as i am guessing based on your little bits of info what you did
<sigipa>
I'm not trying to be vague. I just tried so much stuff that I've kind of lost track of it all.
<sigipa>
It has been a little while since then. I came back to check pony when I saw the release announcement.
<sigipa>
I usually just pull the latest and run a bench test against the http server to see if something has changed.
<SeanTAllen>
¯\_(ツ)_/¯
<SeanTAllen>
Im not sure that is the correct approach
<sigipa>
If I understand what you are saying correctly, the problem with the http server is probably that it is maintaining references to actors longer than necessary?
<SeanTAllen>
But I dont really understand the problems you are running into
<SeanTAllen>
Its quite possible, no one has fully dug into the issue
<SeanTAllen>
Its possible that quiescance isnt triggered before an OOM
<SeanTAllen>
The normal behavior for quiescance is to attempt those checks when a scheduler isnt doing "normal" work
<SeanTAllen>
that means if you were to keep the server incredibly busy, that even if you arent holding the references, they might not be reaped
<sigipa>
The best description I can give is that if I create a few actors and put them in a loop, things are fine. If I try to create actors on demand, the process will eventually run out of memory.
<SeanTAllen>
If I knew in decent detail what your design looked like, I could give you a better idea of what your problem might be and things that would have to change in either the design or in the runtime to work for that use case
<SeanTAllen>
That's not enough information for me to go on sigipa.
<SeanTAllen>
If you could create some simple code that is how you are doing the design, what the design seeks to accomplish, and are available for conversation, I think we could start to address the issue
<sigipa>
Right now the components in the piple are static. The reason I am interested in pony, is to be able to create components on the fly.
<sigipa>
pipeline
<SeanTAllen>
I'm happy to try and help if you can provide the information I need. My general advice would be if you are trying to do something that is very high performance, you are going to need to understand internals at this time.
<sigipa>
You have given me some things to think about. The gc calls may also help. I will give it another shot.
<SeanTAllen>
We've built a high-performance, low-latency stream processing system in Pony and have a good amount of experience.
<sigipa>
Ok. I appreciate your willingness to help. I will try to do what you recommend and come back with the results.
<SeanTAllen>
Those arent recommendations
<SeanTAllen>
They are "here are some tools you have"
<SeanTAllen>
I dont have enough information to make any recommendations
<SeanTAllen>
I just want to be clear on that
<sigipa>
I meant recommendation in the sense of what I inferred from your comments.
<sigipa>
I need to look at whether actors may be lingering die to held reference and I need to try invoking the gc to see if that helps.
<sigipa>
due
plietar has quit [Remote host closed the connection]
plietar has joined #ponylang
<SeanTAllen>
ok
<sigipa>
Thanks for your time Sean. Have a good day.
_andre has joined #ponylang
plietar has quit [Ping timeout: 258 seconds]
plietar has joined #ponylang
plietar has quit [Remote host closed the connection]
plietar has joined #ponylang
plietar has quit [Ping timeout: 258 seconds]
<SeanTAllen>
you too sigipa, hope some of that helps