<bimawa>
Where I can find job like pony lang developer )))?
acarrico has joined #ponylang
jemc has joined #ponylang
endformationage has joined #ponylang
<rirc_F21E>
bimawa: at our company
<rirc_F21E>
bimawa: roughly where do you live
<bimawa>
rirc_F21E: in Russia, Ural.
<rirc_F21E>
bimawa: hmm, too bad. Unless you want to move to the UK.
<bimawa>
rirc_F21E: remote position not possible?
<rirc_F21E>
bimawa: it's difficult outside of the UK because regulations prevent certain data we work with from leaving the country
<rirc_F21E>
but give it a shot anyway.
<bimawa>
rirc_F21E: i need work visa?
<rirc_F21E>
I don't really know. Send an email here with your CV and questions: careers@phpartnership.com
<rirc_F21E>
And you'll find out.
<bimawa>
rirc_F21E: ok thanks a lot, I'll send it later.
<Candle>
Ha, Newport IoW. just a "short" swim from where I used to live.
khan has joined #ponylang
gokr has joined #ponylang
<rirc_F21E>
I think I may have figured out my odd error message from yesterday. It called the actor field a parameter because it was captured in a closure, is that possible?
<rirc_F21E>
I'm dave24 by the way.
<jemc>
rirc_F21E: I'll have to look back in the IRC logs to see your chat from yesterday, but yes, that sounds right (though not obvious)
<jemc>
a lambda gets compiled to an object literal, which in turn is compiled as an anonymous class
khan has quit [Quit: khan]
<jemc>
lambdas can sometimes be a bit clunky to manage - I've pushed some major UX improvments into them, but they are still somewhat surprising from time to time
<rirc_F21E>
it was actually an object literal in this case.
khan has joined #ponylang
<rirc_F21E>
But that means it's not a compiler bug, yay! Just me as usual.
<jemc>
so, your object literal is an actor? that is, an object literal with some `be` behaviours in it?
<jemc>
or it's created inside of an actor?
<rirc_F21E>
no, the object literal is not an actor, but it is passed to another actor. The object literal defines the callbacks.
<rirc_F21E>
and the callback accesses a field from this actor that is not sendable
<rirc_F21E>
which means the other actor is trying to write to a field in this actor via a function in the object literal.
khan has quit [Client Quit]
khan has joined #ponylang
<jemc>
so, if you can't share your code, I'll ask a few more questions to help you hone in on the right pattern
<rirc_F21E>
this khan fellow just can't stop quitting, huh?
<aturley>
rirc_F21E that'll do it
<rirc_F21E>
jemc: I fixed it already, I realised the access should not have been in the object literal. It was just a refactor mistake.
<jemc>
great
<rirc_F21E>
Thanks though!
<jemc>
no problem :)
<aturley>
rirc_F21E i'm glad you found that. sound's like a tricky one to hunt down.
<khan>
sorry if my crappy network messages bother you folks. I just like to hang around
khan has quit [Quit: khan]
khan has joined #ponylang
khan has quit [Read error: Connection reset by peer]
<SeanTAllen>
no worries khan
<SeanTAllen>
not sure what you are referring to.
<jemc>
(if quit messages bother anyone, it might be useful for them to check their IRC settings and hide quit/join messages)
<SeanTAllen>
o, ya i have that hidden. that would make sense for what he was talking about.
<SeanTAllen>
me :facepalm:
<jemc>
yeah, I have it configured to only show the join/quit messages if that person had a chat message in close time proximity to their join/quit
<jemc>
no problem - thanks for helping out with reporting the issue
SenasOzys has joined #ponylang
smuggle has joined #ponylang
<smuggle>
Hi! I have some questions about FFI
<jemc>
smuggle: welcome!
<jemc>
feel free to ask away
<smuggle>
So I want to make some lightweight MIDI bindings. I figure I'll use JACK since it's suitable for real-time stuff and cross platform
<smuggle>
The tricky part is that the docs say that any callbacks need to avoid allocations/locks/mutexes
<smuggle>
Obviously it'll require extra runtime in C to wrap ringbuffers
<jemc>
yeah, JACK will probably be a tough one - in general it's a bit hard to integrate into pony a library that wants to be in control of the event loop
<smuggle>
Okay, thanks, that was my general impression
<smuggle>
I think if I spawn a thread specifically for JACK and use a couple queues, I can prevent (de)allocation in the real-time thread
<jemc>
yeah, I think you're on the right track, but you'll probably be treading on new ground - I've done thought experiments before about such a system for dealing with such libraries, but I'm not familiar with any substantial work done like this in the past
<jemc>
I'd definitely be interested in seeing what you come up with, and trying to facilitate where I can
<smuggle>
Cool, if I get anywhere interesting I'll be sure to share
<smuggle>
The other half is that I need to signal Pony code in a real-time way from the JACK thread
<jemc>
it's possible that we need a new abstraction in the Pony asynchronous IO runtime to handle interfacing with other native threads like this
<jemc>
yeah
<smuggle>
Because I guess I could have a duty-cycle timer checking something atomic, but that seems wrong
<jemc>
have you looked at all yet into how I/O actors work in Pony?
<smuggle>
Only skimming the docs after reading the paper and a bit of the manual. The types are a bit overwhelming :/
<jemc>
I can walk you through the basics real quick
<smuggle>
That would be great! Give me a minute to digest and cross-reference?
<jemc>
I'll just talk at you a bit, and then you can take your time to digest and ask whatever questions you want: so, the Pony runtime has an asynchronous I/O engine tied into the actor system (called ASIO)
<jemc>
(similar to libuv, for example, if you've worked with that before)
<jemc>
the basic idea is, you have a file descriptor in hand, and you (as an actor in Pony) pass it to the ASIO engine and say "call me back when this file descriptor is readable" (or writable, depending on which flags you choose)
<jemc>
the ASIO engine will use a platform-specific implementation of trying to efficiently wait for those file descriptor it holds to enter the desired status (IOCP on Windows, kqueue on MacOS, epoll on Linux)
<jemc>
then that actor can read from that file descriptor, and do whatever else it needs to do
<jemc>
so, if you had a C thread holding a file descriptor, and a pony actor holding the other end of it, you could signal the pony actor by writing some data to it
SenasOzys has quit [Ping timeout: 248 seconds]
<smuggle>
So ASIO is basically a cross-platform select(3), hooked into the Pony scheduler?
<jemc>
yeah
<smuggle>
What's a noisy event?
SenasOzys has joined #ponylang
<smuggle>
Er, subscription?
<jemc>
so, you may have noticed that you don't have to tell a Pony program when to exit
<jemc>
even if you have work that gets passed around among many actors before getting complete, the runtime knows when you're actually done doing work
<jemc>
this is because it automatically detects *quiescence* in the overall system of actors - the runtime will exit when no more messages are possible
<jemc>
but if you, for example, opened a TCP listener socket, there will always be more messages possible, until that socket is closed
<jemc>
so we say that such a socket is "noisy" - it keeps the runtime from reaching quiescence
<jemc>
so all noisy event subscriptions must be unsubscribed, as a precondition before the program can exit
<smuggle>
I see, thanks
<smuggle>
Thanks, that was really helpful
<smuggle>
I see that timers are also handled via file descriptors
<jemc>
yeah
<smuggle>
Wonder if there's a way to queue a file descriptor signal without a system call
<smuggle>
Although, now that I think about it, a cross-core cache miss would probably dominate the time anyway
<jemc>
_rck: I did some snooping on your github profile after you filed the issue ticket and I notice you're into CRDTs - so am I!
<jemc>
let me know if you want to chat about CRDTs sometime
<jemc>
heh, whoops - I forgot we already talked briefly about both of those earlier this week - I forgot your username since then, sorry for the noise
_rck has quit [Ping timeout: 256 seconds]
<smuggle>
Okay, I was being too precious with the whole "real-time" thing. Lots of JACK examples use pthread mutexes to signal other threads
<smuggle>
So it wouldn't be crazy to have 1. the JACK event loop; 2. a thread receiving signals and sending them to 3. the Pony scheduler
<SeanTAllen>
your choice of "precious" as the descriptor made me laugh. thank you for that smuggle .
<smuggle>
Sure :)
<smuggle>
Even better --- is there a mutex somewhere in the Pony runtime that could release a message?
<jemc>
can you clarify that question?
<smuggle>
Um, not yet, let me think
<jemc>
(though the general answer to the question "is there a mutex somewhere in the Pony runtime" is "no", and that's usually a bragging point :D)
<SeanTAllen>
there are some atomics, but other than...
<SeanTAllen>
there's a little bit as an option for dynamic scheduler thread scaling but thats not what you are looking for
nisanharamati has quit []
<smuggle>
I guess I'm just looking for a way to queue a message, without calling an allocator
<smuggle>
Could steal pre-allocated ones from a lock-free queue, but that's pretty intrusive into the runtime
<SeanTAllen>
what do you mean by "without calling an allocator"? do you mean like malloc? pony has a pool allocator so odds are you arent causing an allocator, but you might call that "calling an allocator".
_rck has joined #ponylang
<smuggle>
The potential issue is needing another page from the OS
<smuggle>
If I can get this project off the ground, I'll be absolutely thrashing memory in the Pony thread, but I'll still want predictable performance in the JACK callback
<smuggle>
It would be better to *fail* to queue a message than to miss a callback
<smuggle>
Does that make sense? Or am I being too picky?
<SeanTAllen>
i have no context for answering that question
<smuggle>
Fair enough
<jemc>
do you have a way stack-allocate (or otherwise statically allocate) a pile of N messages on the JACK callback side, and hold onto them while the Pony side uses a pointer to each message, then returns it back to your JACK thread, so you know its free to use again?
_rck has quit [Ping timeout: 260 seconds]
<smuggle>
That seems possible. Doesn't Pony free everything in its message queues?
<SeanTAllen>
no, it frees the things it allocated.
<jemc>
in order to make things safe, you'd have to have the discipline on the JACK side to not touch the memory for the messages at all while they are marked as "in use", then when you get a message back you'd mark it as no longer in use
<jemc>
and on the Pony side, you'd probably want to treat the message as an `iso`, and require that the `iso` comes back to send to the JACK side to guarantee that no other pony actor has retained a readable reference to it
<strmpnk>
Round tripping an iso refcap is a good idea if you want to avoid GC overhead. I've considered this a few times for accelerating entity component systems which need to allow multiple non-overlapping systems to iterate over shared data quickly.
<SeanTAllen>
with the current gc, there's overhead for that strmpnk
<SeanTAllen>
but, that is going to be addressed
<jemc>
to pass the message from JACK to Pony, you could write the pointer/address to a socket, so that pony would get a read event on the file descriptor, and be able to read the pointer address - then you'd need to make an FFI call to some library function to convert the point address into a struct pointer
<strmpnk>
SeanTAllen: This is related to cycle detection?
<SeanTAllen>
no, orca messages
<strmpnk>
Ah, you mean whichever actors touch it will send their counts back along the chain. Okay. Yes, but I'm not sure the descriptor based flags are really a great cure for this case since we'd now have to box everything we pass through the iso.
<strmpnk>
Or perhaps there is a different change you have in mind for addressing this.
<SeanTAllen>
not sure what you mean by "descriptor based flags"
<strmpnk>
Using bits in the object descriptors to handle reference counts to avoid ORCA overhead.
<SeanTAllen>
what do you mean by "now have to box everything we pass through the iso."
<strmpnk>
The pointer passed would have to be boxed (or have bits reserved for use by the GC).
<SeanTAllen>
i guess i dont know what you are talking about
<SeanTAllen>
"Round tripping an iso refcap is a good idea if you want to avoid GC overhead. " <-- apparently i dont know what you meant by that
<strmpnk>
If I have something I'm managing in C rather than pony, I can give permission to Pony to read/write using iso. Passing it back allows me to pass it somewhere else safely or free it. You can create things like safe parallel iterators over arrays for example. The C code has to only track where things are checked out and never let them overlap.
<SeanTAllen>
ok, that wasnt what i thought
<SeanTAllen>
carry on
<SeanTAllen>
ignore everything i said
<strmpnk>
It's useful for stack allocated things too since you can require that a call returns an iso of an opaque type (like an authority).
<strmpnk>
I haven't tried that one yet but you could emulate a sort of borrowing with it.
* strmpnk
started considering this when looking into getting something like Erlang's ETS but without locks into Pony.
<smuggle>
What is an application message?
<SeanTAllen>
a message sent from one actor to another
<smuggle>
In contrast to a message generated from an IO event or something?
<SeanTAllen>
non of these are application messages:
<aturley>
the problem we're running into is that that causes merge conflicts, and so when we then try to do the `_CheckoutTag` that fails because we haven't resolved the conflicts.
<jemc>
so you're making independent local changes to the git repo in the deps folder?
<SeanTAllen>
no
<aturley>
well, no ... sorry
<aturley>
i should have been more specific.
<SeanTAllen>
bad aturley !
<SeanTAllen>
no biscuit!
<aturley>
we're using a specific tag of the dependency.
<aturley>
so not master.
<aturley>
so we aren't making any changes ourselves.
<aturley>
but we have a non master version in there, so when we run `git -C [root] pull [url]` it basically tries to merge master into what we have in `.deps`.
<jemc>
got it
<SeanTAllen>
and so, dear reader, we have come to the root of Sean's dastardly deed.
<aturley>
i've replaced the command with `"git -C " + root_path() + " checkout master && " + "git -C " + root_path() + " pull"` and i can now call `stable fetch` multiple times without issue.
<aturley>
strmpnk has suggested that there is probably a more efficient way to do this that doesn't involve replacing the working copy of the files.
<jemc>
> In its default mode, git pull is shorthand for git fetch followed by git merge FETCH_HEAD.
<strmpnk>
It's possible to read out the tree for a given ref and update a bare repo but it really depends if the .deps copy should be something fully managed or modifiable by devs. The prior would be a good time to use a shared cache folder (i.e. XDG standard).
<SeanTAllen>
i think full managed
<aturley>
so could i basically do `"git -C " + root_path() + " fetch"` instead of `git -C [root] pull [url]`?
<jemc>
I think so, yes
<jemc>
though I haven't tried it yet
<strmpnk>
I would definitely have to reread some docs to get the commands right but I might look into it then.
<aturley>
that's ok, i'm on it :)
<SeanTAllen>
i smell a pony stable release coming
<aturley>
SeanTAllen kind of like the pony stable release we smelled last week at tech day?
<jemc>
strmpnk: there has been some talk in the past about using a shared cache folder with local symlinks
<SeanTAllen>
i dont remember that aturley
<SeanTAllen>
o
<SeanTAllen>
god
<SeanTAllen>
i get it
<SeanTAllen>
wow
<SeanTAllen>
that was fucking horrid aturley
<aturley>
I AM MAKING A POOP JOKE
<SeanTAllen>
what a awful joke
<SeanTAllen>
i now
<SeanTAllen>
know
<SeanTAllen>
i got it
<aturley>
NO SEAN I AM MAKING A POOP JOKE LAUGH PLEASE
<jemc>
cquinn was working on a big research effort to collect requirements and ideas for a successor to pony-stable
<aturley>
sorry
<strmpnk>
jemc: Yeah. Its a rabbit hole but it's git... so it'll involve spelunking either way.
<strmpnk>
I've warmed up to how rebar3 manages things (esp the checkout option for easy development of deps along side project code).
<strmpnk>
They don't use caches for git repos last time I checked though. So many the one minus.
<strmpnk>
So maybe*
<jemc>
I've been thinking a bit more about what a robust dependency management system would look like, and I think it would involve adding some new hooks to ponyc to allow the dependency manager to resolve package directories using not just the name of the package, but also the directory of the package that is asking for it
<aturley>
welp, i'm elbows deep in some stable stuff right now, and this seems to get me to a better place, so i won't let the great be the enemy of the good, at least locally.
<aturley>
i'll open a PR and smarter folks than me can decide if we merge it in or we try to do something better.
<aturley>
thanks jemc SeanTAllen and strmpnk
<strmpnk>
aturley: 100%
<jemc>
theoretically each package should be able to have its own `bundle.json` that says how its dependencies will be resolved, *without* affecting resolution for *other* packages
khan has joined #ponylang
<jemc>
pony-stable only reads one local `bundle.json` in the primary compilation directory, and the `PONYPATH` changes introduced by it will affect all other packages, in ways that are possibly very bad when names/versions start clashing
<strmpnk>
jemc: I think I disagree on the "without affecting" part. It leads to needing recursive upgrades when one dependency needs adjusted. Lots of small commits across many repos.
<SeanTAllen>
yeah, stable is very simplistic and ultimately stop gap
<SeanTAllen>
jemc: i think you are definitely right with everything you said
<strmpnk>
The most important thing is to have a very clear way that dependencies and versions are resolved. After having dealt with npm, gems, and others a lot, I very much am against version operators with local only resolution.
<strmpnk>
Ignoring the DAG when resolving versions is another mistake. The top-level should take precedence. If it doesn't match nested requirements, then a warning message should be given, not a resolve error.
<strmpnk>
This helps when there are common cross cutting concerns that need to be upgraded and deps are older and haven't clarified their compatibility.
<strmpnk>
Semantic versioning doesn't help much here since we don't write bug free software 100% of the time, so top-level overrides always give targets the ability to have obvious workarounds.
<strmpnk>
Anyway... I'll stop going on about this for now. :-)
<jemc>
strmpnk: so, if I understood you correctly - you're more or less in agreement with what I said, but with the addition of global overrides?
<strmpnk>
Not global but DAG order.
<strmpnk>
I'd recommend checking out how rebar3 does dep resolution.
<jemc>
okay, DAG order makes sense, I think
<jemc>
I will check out rebar3 at some point
<strmpnk>
(it's build dir is also nice but that is more of a ponyc concern which won't matter till we look into multi-translation unit builds)
<smuggle>
jemc: You said "...then return [an allocated message] back to your JACK thread..." (i.e. once it's been handled). How can I return a message once it's been processed?
gokr has quit [Ping timeout: 240 seconds]
<smuggle>
Oh, I think I found it, (_pony_type_t).dispatch
<jemc>
I think the answer depends on how you can get an event with data into your JACK-controlled event loop
<jemc>
I'm not super familiar with the JACK event loop