<avsm>
starting with the canopy discussion from last week -- looks like its live now mato!
<avsm>
tell us more...
thomasga has joined #mirage
<avsm>
any drama in the setup?
magnus_ has joined #mirage
<avsm>
...or hannes if mato has disappeared.
copy` has joined #mirage
<hannes>
I don't know anything about canopy setup, sorry
<yomimono>
I asked mato whether it was worth writing a blog post about and he said not really, it was a fairly standard xen deployment
<mato>
yeah
<yomimono>
(sorry to speak for you mato)
<avsm>
oh, so its not running on solo5?
<mato>
there's nothing particularly interesting about it, all the specifics are to do with my dom0
<avsm>
ok! at some point I'll gather all the infrastructure under mirage.io into one place, but its great to have it for now
<mato>
not at the moment since i wasn't sure what state hannes' network access was and the only machine i have with public ipv4 addresses spare is a xen box
<avsm>
I hear that the original purpose of it (for GSoC) has been successful, and that we got one GSoC student accepted
<avsm>
is that confirmed thomasga?
<yomimono>
gsoc hasn't announced acceptances yet
<yomimono>
and they're QUITE strict about not doing it before you should
<thomasga>
nothing is public yet, still disccusing about it
<yomimono>
so you won't hear about that until the student does
<yomimono>
assuming there is one
<avsm>
oh ok, so just wait for confirmation on that
<avsm>
The next item is about moving more infrastructure onto a Mirage3 base, specifically DNS
<hannes>
avsm: is centralising infrastructure of mirage.io a goal for you?
<avsm>
hannes: centralising information about the mirage.io infrastructure is a goal for me, not the infra itself
<avsm>
specifically who runs what, and is responsible for it
<hannes>
avsm: we talked last time that we could run our very own ocam-dns as authoritative nameserver -- there are some IPs and machines available -- after talking with mato y'day in a pub, I thought I'd just bring this up here again
<avsm>
definitely would like to. The last time I set this up it was just lacking an effective update story, and I was hoping that we could have a canopy-style setup where ocaml-dns-server clones off a GitHub repo for its zonefiles
<mato>
+1, and i'm happy to document anything that's set up, though some of that may need a private repo (not sure)
<hannes>
specifically if we can get a zone dump, we can get started and then, once the setup is there, migrate from ghandi to our NS... (NB: I run NS for nqsb.io using ocaml-dns since months without problems)
<avsm>
i think we have all the pieces now to drive mirage.io DNS off a github repo, just like canopy -- wdyt?
<hannes>
I would not integrate any unneeded git code and push hooks in there, just use a zonefile in a git repo and recompile on the other end whenever that changes..
<avsm>
if you want to create a mirage/ns.mirage.io repo and put a test zone file in there, i can send a PR with the full zonefile
<hannes>
avsm: will do this week
* mort___
curses broken irc client and says hi quietly
<avsm>
awesome! and hello mort :-)
<avsm>
If anyone else would like to deploy infrastructure on their own nodes, please do feel free to step in. It would be great to have DNS in particular spread out among as many backends as we can find
<avsm>
This does bring us up to the next agenda item, which is a couple of pressing bugs in two libraries: TCP/IP and DNS.
<hannes>
I implemented a recursive resolver and run into trouble ; TL;DR: is anyone maintaining ocaml-dns?
<avsm>
The library is super old, and needs an update to all the nice modern libraries we have. It is increasingly difficult to maintain it as a monolithic opam package with lots of optional dependencies
<avsm>
split it up into multiple OPAM packages, and then modernise the parser/printer in particular
<hannes>
avsm: the problem imho is already in the "core", sorting out subpackages won't help there much...
<yomimono>
I think avsm's proposal is "why not both"
<avsm>
hannes: it helps a lot, since we can just fix and rev the core without having to fix the lwt/async/mirage backends all at once
<hannes>
it uses exceptions all over, hashtables, ..., does not comply with RFC1025
<djs55>
I think both are problems: the core APIs aren't good plus I got really confused by the depopts when I last tried to fix and test something
<avsm>
so i suggest doing the opam breakup first, fixing all the revdeps with upper bounds, and then having the freedom to make an exception-free parsing core
<avsm>
mort___ : istr you touched the core last with the big cstruct port...
<hannes>
(and I actually have by now an exception-free parser of DNS frames, and a sensible Name.compare)
<mort___>
wow, is it that long since anyone touched it? :)
<mort___>
maybe
<avsm>
it has been some time :-)
<avsm>
i can start the build/revdep breakup after we finish the cstruct/cohttp one i think
<avsm>
and djs55/magnuss have offered to look at the parsing core after, and I think it could use some AFL magic as well while we're there
<mort___>
hannes: if you already have fixes that would be appropriate to apply, what would be the easiest way to take those in? before or after the breakup avsm suggests? (which i agree is necessary to do)
<avsm>
good question...should we apply some emergency fixes?
<hannes>
mort___: well, I removed the bits I don't care about. pretty sure there are modules which do not compile anymore
<djs55>
The code works pretty well given there are still compliance bugs in it. I'm not sure anything is urgent?perhaps we need to schedule some time to read through the RFCs carefully too and write tests
<avsm>
hannes: i'd be surprised if something doesnt actually compile in there -- everything should be a depopt somewhere
<avsm>
it's just a maze of optionally activated findlib modules
<mort___>
true. maybe better to break it up first and then (carefully? :) replace parsing code with hannes' modern versions?
<avsm>
agreed.
<hannes>
sorry, I stole pieces of name and packet, and removed the bits I do not need. I might put the code at some point somewhere
<avsm>
it's a good chance to look at using angstrom perhaps, and benchmark it+flambda
<avsm>
hannes: understood.
<avsm>
thomasga: i wonder if some of romaine's experience with git might come in useful here
<hannes>
djs55: this depends on the use case -- as mentioned, I implemented a recursive resolver, and run into lots of trouble
<avsm>
thomasga: (from the perspective of making a fast parser that's also flambda friendly)
<djs55>
I'm personally less interested in performance than correctness (for my use cases)
<hannes>
(such as requesting for an a record for ssl.engineering.nyu.edu which returns name in all caps)
<rgrinberg>
For something like dns, won't bitstring work better?
<avsm>
rgrinberg: ironically, the original version used bitstring
<mort___>
i'm not putting bitstring back in :)
<mort___>
took it out once, that was enough
<djs55>
I think the use of hashtables in the label decompressor is a bit hairy (and independent of cstruct vs bitstring)
<mort___>
(istr it wasn't as efficient as expected either, or at least there was some perforamnce surprsie. can't recall…)
<mort___>
djs55: agree
<rgrinberg>
Interesting. So bitstring is to be avoided or it just didn't fit here?
<avsm>
yes the issue with bitstring is manipulating string fragments on the heap (lots of them)
<thomasga>
avsm: I don't know anyone called romaine :p
<hannes>
less loopy, no hashtables, also allows pointers to Z :)
<avsm>
rgrinberg: i think there was a port of bigstring to use cstruct, but then you run into issues if it comes in via two cstructs due to an IP fragment. So I'm wondering if writing a 'cstruct list' parser for Angstrom would be better, as we can then write the DNS parsing in a nice combinator style
<djs55>
hannes: looks much nicer to me!
<hannes>
as djs mentioned on github, maybe there should be a separate library dealing with Name.t...
* mort___
sheds a tear for the long past feeling of satisfaction achieved when he finally fixed the pointer parsing to always terminate (or so he believes), while also being unable to argue against hannes' code being much nicer
<avsm>
Yes I like dsheets' suggestion of minimising the dependency cone of ipaddr, so we just have a dns-name opam package
<avsm>
this tcpip one cropped up for good reasons: with more backends, we are now getting more interesting ethernet interfaces to send traffic over!
<hannes>
the TCP-IP issue... I'm curious how nobody else discovered it in a package released since a month...
<avsm>
because it doesnt happen on xen, right?
<avsm>
only on virtio
<yomimono>
`it` being the crash, not the wrong behavior
<avsm>
yeah
<hannes>
avsm: maybe now its time to look into what mirage-net-xen actually does if it receives a frame > its mtu!?
<avsm>
s/mirage-net-xen/xen :-)
<hannes>
avsm: well, IIRC mirage-net-xen has page size / 2 for every outgoing frame
<djs55>
if the frame makes it to the backend I expect it'll be silently dropped in dom0
<avsm>
its up to the linux dom0 bridging stack, as its just a feature flag iirc
<hannes>
and I don't remember what it attempts to do with oversized frames
<avsm>
yomimono: sounds like you're writing tests so you've got the TCP issue in hand wrt a release?
<avsm>
(I'm trying to move onto your agenda item before your battery runs out :-)
<yomimono>
avsm: yes.
<avsm>
ok! next up: issue runthrough and tagging party on 9 May 2017, time TBA yomimono
<avsm>
tell us more!
<mato>
yomimono: is this a 3.1 release in disguise? :-)
<yomimono>
amirmc and I are giving a MirageOS 3 talk at OSCON and I think it would be nice to clean our house up a bit before we (potentially) have people over
<yomimono>
mato: no, when we do a 3.1 release you'll know about it ;)
<yomimono>
I'd like to coordinate a time for maintainers to run through issues in the family of repositories we have
<hannes>
yomimono: sounds great. should we have some place to collect feature requests (such as "revive freebsd kernel backend" etc., which are filling the github issues?)
<avsm>
sounds great to me. could we standardise on our issue labels?
<yomimono>
hannes: yes! and I think that place is canopy
<yomimono>
I'd like to create a few more homes for things smaller than pioneer projects
<yomimono>
small bugfix/feature requests and intermediate ones
<amirmc>
+1 for things smaller than pioneer projects
<yomimono>
avsm: yes, I'll send out a mailing list message about labels
<avsm>
sounds good to me, in my calendar
<yomimono>
the reason for a specific date is that it's very easy to put this kind of thing off until later, and I find it useful to have social pressure to do it with others
<avsm>
absolutely, and I think it'll relieve a lot of ambient pressure of ancient issues weighing down on our repos
<yomimono>
in the absence of others' input about specific timing I'll just announce it at a time that's convenient for CST
<yomimono>
er, CDT, sorry, which is UTC-5
<avsm>
UK afternoon is good for me
<hannes>
it is unlikely I'll have time on 9th -- meeting with justin cappos during that week... but I'll see what I can do
<yomimono>
cool, thanks hannes and avsm :)
<amirmc>
And it's the day before the talk so there's immediate value for anyone who drop by afterwards — just so you know your efforts matter :)
<avsm>
if anyone objects, speak up there please. but i'd like to push this one through even if it inconveniences a few PRs, since otherwise we lose history on our contributors
<hannes>
is it correct? (as in, the HEAD of it compared to the current HEAD is equal)?
<avsm>
thomasga?
<avsm>
i havent diffed the two branches personally
<avsm>
(Cant do it via git as they have different roots)
<yomimono>
it's missing a few recent commits (sorry thomasga)
<avsm>
ok, so it's ok after a last `diff` sanity check across both branches i guess
<avsm>
Last item: good news on the infrastructure front -- we have more x86 machines and also impending access to ARM64 (96-core packet.net) and POWER (64le/be) thanks to IBM (yay djwillia!)
<avsm>
this is in preparation for an overhaul of the OCaml/OPAM infrastructure, particularly with OPAM2 and new features like hannes' conex on the horizon, but it should also be usable by Mirage
<hannes>
does mirage run on power?
<avsm>
No, but now it can... this is just in time for the ARM64 merge of solo5, so we can CI that as soon it lands
<avsm>
OCaml runs on POWER but is struggling with CI, so that's obviously a prerequisite before a Mirage port
<avsm>
Not sure how interested djwillia is in a Solo5/Mirage POWER port
<avsm>
but we have the machine coming in soon so yay :-)
<mato>
This'll be a physical machine somewhere?
<avsm>
sort of, it's on an IBM cloud
<avsm>
depending on what we need its either physical or a VM
<avsm>
but we should get serial
<mato>
ack
<avsm>
that's pretty much it for the agenda. was there anything else to bring up?
<hannes>
oh, related to infrastructure: I went through buerocracy and got an object identifier from IANA registered to Camelus Dromedarius :)
<avsm>
ah yes! excellent name...
<hannes>
in case someone wants a sub-OID, please let me know
<hannes>
(OIDs are used in ASN.1 based data exchanges: SNMP, X.509, LDAP, X.500, ...)
<djwillia>
(avsm POWER would be cool - I have no experience with it but people here at IBM say it's an awesome HW platform)
<thomasga>
(for the mirage-tcpip rebase, I posted the recipe on the issue: basically I've just changed the first commit in the history to point to the last one on the old branch)
<djwillia>
(but they might be biased :)
<thomasga>
(so it's easy to redo if some commits are missing)
<avsm>
djwillia: we're doing everything to make it easier, but I'm guessing that it'll take someone to drive the final push for a mirage release with POWER to come from IBM or a user
<dinosaure>
rgrinberg: hmmhmm ?
<avsm>
dinosaure: TL;DR considering using angstrom to rewrite the dns parsing core more safely, and wondering about your experiences with building fast code using it in git; https://github.com/mirage/ocaml-dns/issues/137
<hannes>
there's amirmc + yomimono agenda item of talking about mirage at oscon...
<dinosaure>
oh ok, I will look :)
<avsm>
oops, I didnt refresh. Go ahead amirmc yomimono
<amirmc>
I think we already covered it. Spread the love!
<yomimono>
does that mean "RT if u agree"?
<amirmc>
Something like that. :)
<avsm>
i'd just like to say how great yomimono's oreilly profile pic is
<yomimono>
hahaha
<yomimono>
thanks
<amirmc>
Who else has access to the Mirage twitter account? I assume it's cool if I send a tweet from it.
* mato
waves (battery at 3%, need the rest to hibernate...)
<yomimono>
later mato!
<djwillia>
avsm yomimono: I had to look it up and now I'll have nightmares... I don't remember yomimono having so many eyes...
<avsm>
Thanks everyone! That wraps up the week 17 meeting of 2017. May week 18 be kind to you, and see you in week 19. 19 is the fifth happy number and the third happy prime, fwiw
<yomimono>
the better to see all things with
<yomimono>
avsm: :D
<amirmc>
_looks up happy numbers_
<hannes>
avsm: should this infrastructure document you're planning maybe just be a canopy page?
gemmag has quit [Quit: Page closed]
talex5 has quit [Quit: Leaving]
amirmc has quit [Quit: Leaving.]
tiimada has quit [Quit: Page closed]
<avsm>
hannes: not yet, it has financials in it also so private for now
<adamrbk>
when I tried just using it in a unikernel.ml file it throws an unbound module error, but the generated ocamlfind command doesn't have the package included
<adamrbk>
do I need to add something to the config.ml indicating that I'm using that package?