00:07 <DanC> python on cloudabi is fun!

00:07 <DanC> it's sorta my favorite ocap platform these days

00:08 <DanC> linux support lags quite a bit, though. no select/poll

00:16 <DanC> "Capability security for mainstream linux with CloudABI, capsicum" is on my wishlist http://www.madmode.com/2017/ocap-wish-list.html

00:16 <DanC> "As a demonstration, I'm porting ZeroVault to CloudABI using a FreeBSD vagrant box VM. It's pretty fun since Ed fixes my issues within a few hours of when I report them."

00:40 <mokomull> DanC: bus1 looked interesting, but the patchset as of Linux Plumbers Conference was pretty buggy. I should re-test my test case and submit some bugs - do you know if they prefer Github or email?

00:48 <DanC> no, I don't know anything about how they work

01:10 <isd> I look at bus1 and my reaction is "why are these not file descriptors?"

05:47 <zarvox> isd: I think the authors intended to have something that would better handle zillions of objects per process, where fds with their density requirements tend to fall over. Additionally, something where the owner of the cap can forcibly close it, whereas fds are refcounted and only close in the kernel when the last ref is dropped

05:48 <zarvox> there's a few other differences in semantics that make bus1 better than plain fds for capability-based IPC

05:49 <zarvox> also I think you get causal consistency from the kernel, which makes for simpler application code?

05:51 <zarvox> the design looked quite good/worthwhile to me; can't comment on the implementation (but mokomull if you wind up doing something with that testcase, let me know how it turns out!)

05:52 <mokomull> isd: I've heard the same thought and *somewhere* I've got written notes from that being presented at Linux Plumbers Conference. If I come across them I'll dump them somewhere for #sandstorm to look at.

06:09 <isd> I've seen a more detailed discussion of it, and yeah, it's not exactly trivial. You'd have to make it possible to have more fds per process, certainly. But it's one of these things where I feel like every time there's a problem that needs solving, we grow a whole new subsystem & interface instead of trying to extend/adapt existing concepts.

06:10 <isd> So we end up with a monumentous API.

06:12 <isd> The force close thing seems like your standard revoker pattern. Though you probably need some help from the kernel to make that efficient, but it doesn't need to be something that's exposed to anyone who's not poking at it.

06:14 <isd> But, it's late and I'm getting curmudeonly. I'm going to try to sleep.

06:16 <isd> (I seem to remember reading a detailed analysis of the challenges in making this work with fds, and it was mostly a question of getting from here to there, rather than what there needed to look like).

06:16 <isd> okay, really, going to bed. 'night all.

14:06 <ill_logic> Well this is rich. I get "Grain no longer in use; shutting down" in the middle of an upload. 13 gigs in, it decides that it's no longer happening.

14:07 <ill_logic> And then it starts back up, but I guess by then, jquery-file-uploader doesn't want to continue.

14:07 <ill_logic> This thing just doesn't seem to want to happen.

14:07 <ill_logic> Any ideas what's causing this?

14:22 <dwrensha> ill_logic: how is the client accessing the grain? Through Sandstorm's usual framed UI or through an API token?

14:24 <ill_logic> framed UI

14:25 <dwrensha> weird. your browser should be sending keepalives to the grain then

14:26 <ill_logic> how often does it need them?

14:26 <dwrensha> I suppose it's possible that it is so busying sending the upload that it doesn't get a chance to send the keepalives

14:26 <dwrensha> approximately every minute

14:26 <ill_logic> I could try smaller chunks then.

14:26 <ill_logic> how often does it need keepalives?

14:27 <dwrensha> you could try keeping the grain open in a separate tab

14:27 <dwrensha> so that the other tab would be responsible for sending keepalives too

14:28 <ill_logic> I don't like that from a usability standpoint.

14:28 <ill_logic> But I suppose it could at least debug it.

14:28 <dwrensha> do you have a reverse proxy running in front of sandstorm?

14:29 <ill_logic> I don't think so. I just did a normal Sandstorm installation.

14:29 <ill_logic> (I'm developing an application so I'd like to make this work without trouble for users, eventually)

14:29 <dwrensha> I know that nginx likes to buffer uploads, so that the backend of the app doesn't even see them until the client is done sending

14:30 <dwrensha> hm... I suppose it's also possible that the supervisor gets so overwhelmed with forwarding the data that it does not get a chance to service the keepAlive() call

14:31 <dwrensha> I thought we had decent flow control for that nowadays, but maybe it's buggy

14:32 <dwrensha> in any case, this sounds like a bug in Sandstorm, not in your app

14:33 <dwrensha> to debug, it might be interesting to watch activity with `top` while the upload is happening

14:33 <ill_logic> I actually do that, though just checking whether it gets overwhelmed.

14:33 <ill_logic> What should I look for?

14:34 <ill_logic> (I don't know what to look for)

14:34 <ill_logic> (I just have it open, heh)

14:35 <ill_logic> And, should I make a new thread on the dev list?

14:35 <dwrensha> yeah, that or open a github issue

14:36 <ill_logic> right.

14:40 <ill_logic> How often does Sandstorm need to be kept alive?

14:42 <dwrensha> each grain supervisor kills itself if it does not receive a keepalive periodically

14:42 <ill_logic> Do you know the period?

14:42 <dwrensha> I think the period is something like 90 seconds

14:42 <ill_logic> I see.

14:42 <dwrensha> you can find it in src/sandstorm/supervisor.c++

14:42 <ill_logic> So I could see potential for problem. If you miss a single keepalive.

14:43 <ill_logic> I'll watch the sandstorm logs too. I see errors in there but it could be from me messing with things earlier.

14:43 <ill_logic> also what if sandstorm upgrades in the middle of all this?

14:43 <dwrensha> I suppose that arguably your app should be breaking up huge uploads into small chunks

14:44 <dwrensha> upgrades of sandstorm do not kill supervisor processes

14:44 <dwrensha> but they do require reconnects

14:46 <ill_logic> reconnects between? I mean, would the same grain restart thing happen?

14:48 <dwrensha> it might, but not necessarily

14:48 <dwrensha> a new sandstorm frontend process will start and will need to reconnect to the grains

14:49 <ill_logic> would it drop existing uploads?

14:50 <dwrensha> yeah, I think in-progress uploads will fail

14:51 <ill_logic> okay. so that's bad news for my application.

14:52 <ill_logic> hmm... jquery-file-upload has a resume option. I'm gonna look into this.

14:54 <ill_logic> Too small of chunks would be high latency I think.

14:54 <ill_logic> I mean, it would stack up latency.

15:20 <Marcelg> Would anyone be able to help me figure out my DNS for my Hugo grain ? I'm making the entries it says to make but I get an error when I visit the URL indicating that the host must have exactly one TXT record. I have added the one TXT record that Hugo grain said to add in the DNS. If there is someone familiar with this could we talk and ma

15:20 <Marcelg> ybe figure this out?

15:49 <ill_logic> (ah, jquery-file-upload has retries bulit in. I'll see if this solves it.)

18:36 <isd> ill_logic: you could probably hack around this by getting a wake lock from sandstorm. Ideally it would just see that there's activity and handle that correctly, but as a workaround...