avsm changed the topic of #mirage to: Good news everyone! Mirage 3.0 released!
copy_ has quit [Quit: Connection closed for inactivity]
_whitelogger has joined #mirage
hdurer[m] has joined #mirage
argent_smith has joined #mirage
mort___ has joined #mirage
Ttime has joined #mirage
Ttime_ has joined #mirage
Ttime20 has quit [Ping timeout: 248 seconds]
Ttime_ is now known as Ttime20
Ttime has quit [Ping timeout: 268 seconds]
mort___ has quit [Quit: Leaving.]
<lobo>
hannes: ohai. i'm not sure if vmm is for public consumption yet, but i wanted to give it a try and got stuck with a "Cannot find file ./pkg/pkg.ml." error, while installing tls.
<lobo>
hannes: i've started to pin a few packages, because i got "Unbound module X509.CRL" while installing vmm first. here is some info about the pins https://paste.headstrong.de/view/raw/f040cd72
<lobo>
reynir: sorry for just throwing links at you :-) hannes has built some software for deploying mirageos unikernels via tls certs
<reynir>
Ah that okay
<lobo>
ehhm, x509 certs and tls
<reynir>
:)
<hannes>
lobo: and while it is not really meant for public consumption yet, I'm ready for getting feedback (in case there are build or setup struggles)
<hannes>
I know that the whole CA stuff is a bit cumbersome to setup, but it's actually needed (and I provided tools in the repository :)
<hannes>
lobo: the linux part hasn't been extensively tested, but I tried it on the command line and these commands worked ;) (happy to replace with ip, but i never understood ip too well, and there were various versions of ip which do not have the tuntap support :/)
<hannes>
it's unclear to me what to do there best... likely depend on a "modern" ip package which has the tuntap stuff?
<lobo>
hannes: afaik, the iproute2 tools should replace the older commands. but i'm going to verify which version of the iproute2 package has all the required features after vmm compiles. i'll give it another try when i'm at home later today
<hannes>
cool, thx!
<lobo>
if you'll ever have to deal with the ip commands again, there is a good cheatsheet http://baturin.org/docs/iproute2/ . it is unfortunately needed, since the majority of linux manpages just don't contain any usage examples :-(
<hannes>
oh nice, thx!
mort___ has joined #mirage
seangrove has joined #mirage
TImada has joined #mirage
mort___ has quit [Quit: Leaving.]
mort___ has joined #mirage
Ttime20 has quit [Quit: Going offline, see ya! (www.adiirc.com)]
<mort___>
(after someone moved me up the agenda :)
<djs55>
(I guess @mor1 on github == mort__ here)
<mort___>
yup
<mort___>
right, so, dommage
<mort___>
i got bored of waiting for travis to build about a million packages every time i pushed commits, so i decided to containerise building things with mirage
<mort___>
dommage is a script that attempts to do that
avsm has joined #mirage
<mort___>
i ported the travis config for my website over to use it; it seems to work ok
<mort___>
i've done / am doing the same for mirage-decks and mirage-tutorial, updating both to mirage v3 at teh same time
<mort___>
there's a PR in place against mirage-www to use dommage, but there are some outstanding requests / comments made against the pr that i've not addressed yet
* avsm
waves -- on via dodgy mobile connection
<mort___>
(largely, iirc: make a release; write a blog post or other form of instruction)
<avsm>
that would be great. the other thing that would help keep mirage-www working is to activate the 'cron' build in travis
<mort___>
the net effect of this for my site at least was that travis build times are down from ~18min to ~3-4min
<avsm>
that will let it build every day
<mort___>
also i have the same build environment on my mac as on travis so hopefully if/when i get more weird packaging mismatch errors, i can actually track them down
<thomasga>
if we have a cron job, someone will have to look at if it fails or not :-)
<thomasga>
(which is probably a good idea)
<djs55>
travis might send email — IIRC I get some when I break things sometimes
<avsm>
it sends email indeed
<mort___>
travis can be made to send email on failure and on success, independently
<djs55>
@mort__ being able to reproduce the CI failures locally is high on my wish list!
<mort___>
i'll try and write that post later today
<mort___>
as it's been a week or so, i'll probably also tweak the script now i've had a chance to reflect on them
<djs55>
a post sounds like a good idea — partly when I read the PR I didn't understand the overall design… a post would definitely help me understand better
<mort___>
i'll also try and make a release (though i note that we've generally only ever gone from master on ocaml/ocaml-ci-scripts anyway)
<djs55>
ok, shall we move on to the next item? (interrupt me quickly if not)
<avsm>
same here. I'd like to reconcile mort's obvious enthusiasm for dommage with my disinterest in yet another caching layer :-)
<mort___>
(or from some arbitrary set of pins)
<mort___>
djs55: cool
<djs55>
ok I think we can move on to: adding end-to-end tests to functoria @samoht
<mort___>
avsm: do we have many already? if having mirage-www bulid from scratch every time is a positive design goal, happy for the PR to be closed since it's clearly in conflict with that
<thomasga>
So I have added more tests to functoria, while trying to debug an issue with cross-stage persistence of configuration keys
<avsm>
mort___: not sure -- lets discuss out of band once i've read the post. no strong opinion yet
<avsm>
thanks for all the tests thomasga !
<thomasga>
this is able to tests nice things if the output is the same on `mirage configure —help` and `mirage help configure`
<thomasga>
and if config.ml is present or not, etc
<thomasga>
e.g. a lots of stuff which has gone wrong in the past
<thomasga>
(and which is usually hard to test)
<djs55>
thank you for volunteering to write difficult tests :)
<thomasga>
so now I am a bit less afraid to modify functoria :-)
<avsm>
i'm keen to see yallop's patch merged, that cleans up the main.ml output
<thomasga>
yea, that would be great
<thomasga>
I have few new ideas for functoria (including generating multiple unikernels using one config.ml) but nothing really concrete yet.
<thomasga>
and I feel a bit more confident to experiment with it now that there are some tests. Anyway, that's all for my update :-)
<djs55>
perhaps you could describe your ideas in an issue? just to get early feedback
<thomasga>
yes I will!
<djs55>
ok, the next agenda item appears to be: removal of OS module - @djs55 @avsm have patches to eliminate it entirely, but depends on xen -> ocaml-freestanding and removal of old platform code
<avsm>
OS can die now! It was originally a "global module" hack before functoria, but we never fully purged it in mirage3
<djs55>
exactly — I think we've all been chipping away at it
<avsm>
so I took a look at this, prompted by hannes' OS.Env removal PR that has been lingering
<avsm>
turns out there were a bunch of phantom dependencies on mirage-unix that i removed and nothing broke
<avsm>
and the only real user of it is OS.Xs in the xen backend -- which ironically isnt portable. So i've got patches to pull that out into a separate library now that can be explicitly depended on
<hannes>
well, OS still contains the main loop, no? where should that code go? mirage-runtime?
<avsm>
right, the only function left is the `run` function -- that should be explicitly called from functoria
<avsm>
since it knows exactly which backend is in use
<avsm>
there's no need for an abstraction layer
<thomasga>
not sure how easy it is to do this, currently `run` is defined in the prelude where there is not conditionals
<avsm>
if only we had a functoria expert in the room
<thomasga>
but I guess this could be put in a device which depends on the target
<avsm>
yeah that would work -- the final device that triggers the rest
<hannes>
since OS is used indirectly by mirage-entropy (and nocrypto IIRC), what/how is the redesign of the platform-specific bits of these libraries (currently it uses mirage-os-shim)!?
<avsm>
yeah so mirage-entropy and some xen lifecycle stuff came up
<avsm>
i think we also need to extend functoria to add a lifecycle device, and have mirage-entropy depend on those for the startup hooks
<avsm>
that also makes it easier to have service triggers for on-demand unikernels that want to key off suspend/resume events (i'm thinking of qubes)
<thomasga>
could you write up this somewhere? I am happy to try to find a new design where all of this fits
<thomasga>
(I am very happy to not have the OS module anymore)
<avsm>
sure. it got delayed as i thought i'd do a quick upgrade of the mirage-xen backend to ocaml-freestanding, and got dragged screaming into the seventh level of minios header hell
<thomasga>
I can also have a go at implementing the functoria bits for it
<thomasga>
(once we set up on the design)
<avsm>
thanks! i'll write it up
<mato>
Hmm, it's not just "run", there's also the other scheduler functions that go with it.
<mato>
eg. wait_for_work
<avsm>
i've also got a design in mind for fixing the c stub situation, which it turns out is fairly easy once the OS module disappears
<avsm>
mato: none of those are exposed to the unikernel
<thomasga>
@mato: this depends on the backend engine
<avsm>
and if they currently are, they need to go through the device layer
<thomasga>
so you can pick the one you choose at configuration time
<mato>
Not to the unikernel, but the device drivers do use them.
<avsm>
mato: right, but they depend explicitly on mirage-solo5
<mato>
yup
<avsm>
there's no need for an OS.Foo abstraction layer
<djs55>
The xen device drivers explicitly depends on mirage-xen, which is ok
<avsm>
my patches to the mirage-block/net-xen ones depend on mirage-xen explicitly now
<avsm>
jinx :)
<djs55>
ok, so it sounds like avsm is going to do a short writeup and post somewhere? (mailing list?)
<avsm>
issue probably
<avsm>
will mail list to point to it
<djs55>
ok great, and then we can make sure we've covered all the cases
<djs55>
that's probably enough for that agenda item — avsm needs the time to work on the minios headers
<avsm>
im building a version of the docs site with only the latest packages
<avsm>
so there's an opam remote with the right constraints
<avsm>
anyone mind if i switch the docs site over to this smaller one?
<mort___>
what is meant by "only freshest packages"?
<avsm>
and then introducing new packages requires they match the global constraint set in mirage-stable
<avsm>
only the newest versions
<mort___>
oh, you mean only latest releases rather than all past releases?
<avsm>
so e.g.. cohttp.0.99 conflicts with lots of packages atm
<avsm>
no i mean do not just build an older version of a release due to a conflict
<avsm>
i want the latest mirage releases to be documented; not an older version since a package that uses it pulled the opam build back to an older mirage release
<thomasga>
I think that's fine.
<thomasga>
Something that we should solve at one point is where to point at the default documentations for the packages.
<djs55>
so if the latest package can't be installed due to conflicts it is omitted? So it's fully up-to-date coinstallable packages? Seems like a nice incentive to fix your package to get it in the list :)
<mort___>
still a bit confused but probably fine. trying to work out what happens when packages do conflict
<avsm>
if they conflict the tree doesnt build :)
<avsm>
so mirage-stable is intended as a small set of packages that have a reasonable chance of building
<thomasga>
(but this could be decided later, having a stable global location for package documentation is a good first step)
<avsm>
and then we have other clusters (e.g. mirage-block-backends) for libs that use that
<thomasga>
avsm: is it easy to produce a list of packages not in the "fresh" list?
<avsm>
hm
<avsm>
i'll go with "yes"
<thomasga>
and ping the maintainers?
<avsm>
i'm working on a tool that'll possibly make that easier
<thomasga>
cool
<avsm>
i'll update on that later if it works
<demonimin>
will it be a new site, or will some urls break?
argent_smith has quit [Quit: Leaving.]
<avsm>
i think some urls might break
<avsm>
but do shout if you're affected
<avsm>
and can try to fix that
<avsm>
i need to find out how to customise odig/odoc as well
<djs55>
ok, I think that brings us to the end of the formal agenda. Any other topics worth bringing up right now? Important issues? Anything unnecessarily blocked?
<mort___>
one quick thing from me
<mort___>
i have a PR in against mirage/mirage to update the tool so that `-kv_ro archive` works again
<hannes>
avsm: thanks for putting the zone of mirage.io into the issue. I'll get us some NS sorted (once my hardware is reachable again)!
<mort___>
any chance anyone could look at it? (it's pretty small)
<mort___>
it would unblock mirage-decks buildings
<djs55>
mort__: could you ping me on it and I'll take a look
<mort___>
(the combined slide decks are too big for FAT or Crunch)
<mort___>
djs55: will do, thanks!
<thomasga>
@mort___: I have fixed the CI scripts on mirage/mirage, could you rebase to make it green?
<mato>
hannes: can you ping me in the mirage.io NS issue? I don't think I get notifications from it.
<thomasga>
(and I can review it too when it's green)
<mort___>
thomasga: ah, ok sure
<djs55>
ok I think we can declare the meeting over. Back to normal IRC chat ;)
<hannes>
I've also a typo fix open on mirage... in general I don't merge my own PRs in the mirage organisation (and appreciate everybody else who doesn't merge their own PRs, but waits for a review..)
<mort___>
(yup, merging own PRs feels … wrong ...)
<mato>
Quick Solo5 ARM64 status update: I've reviewed Wei Chen's latest changes, then went on a code simplification spree and replaced 300 lines of page table setup code with ~15 + comments. So, once Solo5/solo5#193 is merged the rest should be pretty quick, I have PRs waiting for ocaml-freestanding and mirage-solo5.
<mato>
Using the branches in those PRs I can successfully build and run the skeleton network example :-)
<mato>
(w/o any manual hacks)
<djwillia>
that's quite a simplification! great job mato!
<mato>
djwillia: It basically ditches all the guest page table setup, since (I verified) that KVM on ARM64 behaves the same as on x86.
<hannes>
(and as mentioned on the mailing list, in case you want to deploy MirageOS ukvm unikernels, I can share my resources (physical machine and IPs) now! :) -- see https://hannes.nqsb.io/Posts/VMM later when it is up again ;)
<djwillia>
mato: how much arch-specific stuff remains on the guest side?
<mato>
djwillia: In the "kernel" you mean?
<djs55>
regarding review: I suspect we have quite a long RTT in some places, perhaps because people are busy / miss the notification / (I'm sure lots of other reasons). I wonder if there's a way to maintain some kind of prioritised list of things needing review. I wonder if people with merge power should promise to review someone else's PR before submitting their own, to avoid too much focus on PR creation and not enough on PR review. (Or to fix
<djs55>
a bug before adding a feature, that kind of thing)
<djwillia>
mato: yes, i know we've talked about it in the past but I'm still wondering how close we are to putting all arch setup (except perhaps the I/O stubs) in ukvm
<djs55>
I really like the github "allow edits from maintainers" feature and try to push extra patches where I can rather than ping the original author and go to sleep again
<hannes>
djs55: appreciated. GitHub additionally nowadays has a "request review from XXX" button, where you can ping others to take a look
<djwillia>
mato: we've been thinking a little bit more lately about how hard it would be to put a ukvm unikernel into a SGX enclave and although it's still intel, we would need the guest to entirely run in ring 3
<djs55>
hannes: that's true, that's also a good feature which I under-use
<mato>
djwillia: Not a lot arch-specific that's interesting, basically only the exception handlers.
<mato>
djwillia: Hmm, perhaps we should do a separate catch up to sync our respective plans/ideas for Solo5 going forward?
<djwillia>
mato: and with the ARM simplifications it seems it'll stay that way, which is great
<djwillia>
mato: yes, we've been meaning to, we can coordinate on slack or something
<mato>
djwillia: Ok, let's do that.
<mato>
djs55: Thans for driving the meeting btw, very efficient :-)
<mato>
It's been ages since we finished early ...
<djs55>
heh hopefully it wasn't too quick. It's hard to tell if someone is typing a paragraph or not
<haesbaert>
hi o/
talex5 has quit [Quit: Leaving]
<mort___>
thomasga: mirage/mirage/mirage.opam seems to have a dep on functoria >= 2.2.0 but the latest i can see is 2.1.0. opam pinning to —dev-repo didn't work. what's the correct rune i should use?
<mort___>
(or else i can just update the PR but it'll not even have been built :)
<thomasga>
mort___: pin with a `dev` version
<thomasga>
e.g. `opam pin add functoria.dev —dev`
<mort___>
ah right, thanks
<mort___>
(old age is causing my wotsit to fail...)
<mort___>
djs55 thomasga: ok, have rebased (i think!) and now awaiting what will hopefully be many green lights
thomasga has quit [Quit: Leaving.]
djwillia has quit [Ping timeout: 268 seconds]
TImada has quit [Quit: Page closed]
amirmc has quit [Quit: Leaving.]
ricarkol has quit [Ping timeout: 260 seconds]
mort___ has quit [Quit: Leaving.]
copy_ has joined #mirage
mort___ has joined #mirage
mort___ has quit [Ping timeout: 255 seconds]
miragebot has joined #mirage
<miragebot>
mirage/master fe0e87a Richard Mortier: fix reference to tar library for `--kv_ro archive`
<miragebot>
mirage/master f726f13 Richard Mortier: further update reference and version for `--kv_ro archive`
<miragebot>
mirage/master d255cb2 David Scott: Merge pull request #848 from mor1/fix-kv_ro-archive...