xmn has quit [Read error: Connection reset by peer]
xmn has joined #neo900
<Joerg-Neo900>
wpwrak: AT cmd set is also genuinely available form Gemalto, you just need to register, much like with Silego to get the docs & DE
<Joerg-Neo900>
at least it been that way, didn't check lately
<wpwrak>
oh, that's good. and makes a lot of sense.
<wpwrak>
btw, silego don't require you to register to download docs. you can just skip it. still annoying that they keep asking, but at least it saves them from having to weed out tons of mailinator and such addresses ;-)
<Joerg-Neo900>
I think that's a bug
<wpwrak>
you mean that it keeps coming back ? well, once could certainly consider it as such
<Joerg-Neo900>
we really should fix the datasheet link for modem in BD
<Joerg-Neo900>
I'd think http://web.archive.org/web/*/http://download.maritex.com.pl/pdfs/wi/phs8* is genuine enough
<Joerg-Neo900>
wpwrak: we need a switch for VUSB_MODEM
<Joerg-Neo900>
rationale: we want to be able to shut down modem USB which is accomplished by deasserting VBUS
<bencoh>
when it comes to vbus, I'd rather say "disconnect" than deassert :)
<Joerg-Neo900>
well, since we have no connector in final design, we can't "disconnect" it, we just can cut it, or disable the regulator that provides it (if any, actually we don't have such)
<bencoh>
that's what I meant by "disconnected" yeah
<bencoh>
-ed
<Joerg-Neo900>
and since modem VBUS is a mere sense input, "deassert" seems the right term
<bencoh>
oh so it's not its power source as well?
<Joerg-Neo900>
no
<bencoh>
hmm
<Joerg-Neo900>
hehe
<bencoh>
is that the only way to "shut down" modem?
<Joerg-Neo900>
it only shuts down USB, not modem
<bencoh>
then ... what's the point?
<Joerg-Neo900>
to... shut down USB?
<Joerg-Neo900>
to start ENUM when reasserting VBUS?
<Joerg-Neo900>
and last not least to reduce power consumption of modem for keeping USB active
<bencoh>
that'd be true if you can actually turn in back on without missing, say ... phonecall events
<Joerg-Neo900>
N900 MG MUSB core and ULPI et al eats a buzzing 50mA
<Joerg-Neo900>
well, you can
<bencoh>
supposedly, or actually tested?
Wizzup has quit [Quit: bbiab]
<Joerg-Neo900>
errr
<Joerg-Neo900>
you're aware we have RING signal, *and* UART too?
<bencoh>
voice over uart?
<bencoh>
but no, I wasn't
<Joerg-Neo900>
VOICE??? not over USB at least ;-P
<Joerg-Neo900>
voice is via PCM interface
<bencoh>
oh, okay
<bencoh>
well as long as uart still works it's fine I guess
<Joerg-Neo900>
you can perfectly operate the complete modem without any USB, unless you need datarates higher than what UART allows
<Pali>
maybe you should create similar page like http://elinux.org/N900 to show status of linux kernel drivers for neo900
<Joerg-Neo900>
tbh I have no idea
<Joerg-Neo900>
note Neo900 UG doesn't develop or provide any software
<bencoh>
which is why it would be available on elinux.org and not on, say, ... neo900.org
<Joerg-Neo900>
well, "no idea". Actually I have an idea about what could reasonably expected to be available
<Pali>
such documentation about drivers avaibility should be collected at one place
<Joerg-Neo900>
sure, but not really my business
<Joerg-Neo900>
I'm neither competent nor do I have the manpower resources
<Joerg-Neo900>
generally all datasheets ad thus driver development is warranted except for WLAN/BT module and the PVR
<Joerg-Neo900>
nothing else that lacks any docs / drivers
<Joerg-Neo900>
and since the PVR is exactly same as in N9, we could "steal" the PVR driver from that firmware image if everything else fails
<Joerg-Neo900>
for WLAN/BT TI officially provides closed blob drivers with compatibility wrapper
<Joerg-Neo900>
now you have picked all the knowledge (incl all old any even one new URL) from my brain and bookmark store, to create such a elinux page
<Joerg-Neo900>
what I wonder is if we might need our own Neo900 wiki
<Joerg-Neo900>
for our commumity to contribute such info there
<Joerg-Neo900>
call for volunteers to install and/or administrate such a wiki
<Joerg-Neo900>
UG can provide server space
<Pali>
would be be elinux.org wiki enough?
<Pali>
IIRC raspberrypi has content on elinux.org wiki too
<Joerg-Neo900>
I have no objections, though then it's obviously out of my control
<Pali>
and people will probably easily find them...
<Joerg-Neo900>
actually I'm glad for each service I don't need to administrate and install and backup and...
<Joerg-Neo900>
though otoh eventually it might cause trouble when we / UG don't have full control over that stuff
<Joerg-Neo900>
but to make long story short: I have no clue how to install a wiki on Neo900 servers
<Joerg-Neo900>
so either you volunteer to do that, and I create an account for you and install whatever packages you ask for, or you find another place for that thing
<Joerg-Neo900>
subdomain wiki.neo900.org is no problem either
<Wizzup>
running a mediawiki is a total PITA not just for security, but also SPAM
<Joerg-Neo900>
yeah, I just pondered using http://wiki.openmoko.org/wiki/Main_Page but that's neither under my full control anymore, and also poorly maintained
<Joerg-Neo900>
the goldelico wiki is a PITA by mere UI to start with
<Joerg-Neo900>
then otoh spam issues could be minimized by strictly authenticated-accounts-only edits being allowed
<wpwrak>
(LED) what ? in the block diagram ? pretty much all of them are there ! :)
<Joerg-Neo900>
no, the monitor LEDs are not there, since we have not added then yet. Too trivial for any need to *test* that concept, however I already asked for them for debugging purposes
<Joerg-Neo900>
in proto_v2
<Joerg-Neo900>
your answer was like "you're asking for LEDs at locations that I personally would not be interested in at all" or somesuch :-)
<Joerg-Neo900>
I think for proto_v2 we can't have too many LEDs
<wpwrak>
oh, those LEDs. well, they're really something only you can place to your satisfaction. my idea of what LEDs are useful and what not is rather different from yours. i'm more a test point guy :)
<Joerg-Neo900>
hehe
<Joerg-Neo900>
ok, here we go: we need one for modem VBAT, obviously. We need one for MICBIAS of the digital IHF mics. We should have one for cameras where possible
<Joerg-Neo900>
modem PWR_IND
<Joerg-Neo900>
and/or VEXT
<wpwrak>
what's VEXT ?
<Joerg-Neo900>
modem's power out supply for "application circuitry"
<Joerg-Neo900>
unused by our design, except for monitoring
<Joerg-Neo900>
proposed use is for e.g. level shifters
<Joerg-Neo900>
it's supposed to be straight from master power regulation
<Joerg-Neo900>
in modem
<wpwrak>
(VEXT) ah yes, not even connected
<Joerg-Neo900>
*nod*
<Joerg-Neo900>
excellent candidate for an LED ;-)
<Joerg-Neo900>
hmmm, GPS_ANT?
<Joerg-Neo900>
prolly makes for an interesting optical signal :-)
<enyc>
Joerg-Neo900: sorry, i meant, pcb layout, rather than circuit schematics =)
<Joerg-Neo900>
we'll publish any layout pdf's as soon as there's something tangible
SylvieLorxu has joined #neo900
<enyc>
ok =)
<enyc>
what problems have been encountered with such layout?
<enyc>
can it (mostly) be done with autolayout + manual hints, or does it not really work that way?
<Joerg-Neo900>
none so far that I heard of. Ahycka is working
<Joerg-Neo900>
~tell jonsger about lazyflashing
<Joerg-Neo900>
~tell jonsger about jrtools
<Joerg-Neo900>
anybody around with a stong git hooks foo?
<Joerg-Neo900>
strong, even
<atk>
git hooks foo? those are just scripts ran at certain points in time automatically? What specifically are you trying to get working?
<Joerg-Neo900>
we are doing a filterdiff job on commit hook from one repo to another
<Joerg-Neo900>
wpwrak: ^^^
<Joerg-Neo900>
I don't know what's the particular question, I barely know what I'm talking about ;-) I hope wpwrak will chime in soonish
<atk>
hmm
<atk>
Joerg-Neo900: on another note, you use spamassassin right?
<Joerg-Neo900>
yep
<atk>
Is it pretty good? What sort of volume of spam do you get?
<Joerg-Neo900>
well, I see between 10 and 50 raw / d. of thise generally 1 flase positive per 4 months and 0.5 to 2 false negatives / d
<Joerg-Neo900>
so yes, surprisingly good
<Joerg-Neo900>
though my complete antispam measure set is more than just SA
<Joerg-Neo900>
I also have a few "manual" filters on charset for example
<Joerg-Neo900>
and some of my accounts do server side filtering/tagging too
<Joerg-Neo900>
then I just filer on "subject: [SPAM]..."
<Joerg-Neo900>
or the like
<atk>
Hmm, wouldn't it make more sense to just use an X header?
<Joerg-Neo900>
sure, I do
<atk>
My volume is getting up to 100 or more per day on some occasions and spambayes alone isn't cutting it.
<Joerg-Neo900>
when the provider uses X header to tag
<atk>
ah, you have accounts for different providers?
<atk>
I don't have that issue, I do all my mail handling myself under my domain.
<Joerg-Neo900>
I have maybe almost a dozen accounts, yes
<Joerg-Neo900>
and yes, no SA or other filtering on my own servers
<Joerg-Neo900>
I also don't IMAP
<Joerg-Neo900>
which is a bit of a lazy and foolish combination
<Joerg-Neo900>
but meh!
<atk>
Yeah, lazy is the default MO
<atk>
but sometimes it becomes more effort to be lazy
<Joerg-Neo900>
:nod:
<atk>
like right now I'm getting quite a few DMARC reports because I sent to a big ML
<atk>
so I finally spent a day writing a DMARC report aggregator that I can plumb into my email infrastructure.
<Joerg-Neo900>
right now my hugest PITA is kmail's akonadi and the constant statistical messup it introduces
<atk>
And now I get a nice sqlite database with DMARC reports I can query, but as I've just done that I thought I might go ahead and revamp my email spam filtering
<Joerg-Neo900>
SA is really nice
<Joerg-Neo900>
just don't miss to have daily updates
<atk>
It looks like you can plumb in spamd into an opensmtpd sort of infrastructure
<Joerg-Neo900>
I should note that I didn't security evaluate SA
<Joerg-Neo900>
yes
<Joerg-Neo900>
spamd is a simple unix filter
<Joerg-Neo900>
raw mail in, tagged mail out
<atk>
cool
<Joerg-Neo900>
sorry, afk for a while
<Joerg-Neo900>
I hope wpwrak will chime in whith the hook related questions soon
<Joerg-Neo900>
push hook, not commit hook
* Joerg-Neo900
<- git noob
mzki has quit [Ping timeout: 248 seconds]
cc___ has joined #neo900
<Joerg-Neo900>
atk: are server side hooks identical semantics as local ones? (assuming that there's even such thing like local hooks)
<Joerg-Neo900>
i.e. is post-commit hook on server being executed when a user commits something *into* the server (inbound), while on user's local machine same hook gets executed when user commits something *out to* some remote repo?
<Joerg-Neo900>
well, not 'same hook' but 'equivalent hook' (in local .git/hooks dir)
l_bratch has quit [Quit: Leaving]
xmn has joined #neo900
<atk>
Joerg-Neo900: there's pre-receive, update and post-receive
<atk>
pre-receive is ran when a push from a client is happening
<atk>
it gets a list of references that are being pushed via stdin
<Joerg-Neo900>
wpwrak: ^^^^^
<Joerg-Neo900>
wpwrak: ping
<atk>
update is similar to pre-receive but it's done once for each branch, and it takes input a bit differently
<atk>
there's three arguments it gets
<Joerg-Neo900>
I looked into that one, yep
<atk>
post-receive is done after the entire process is completed, like this is what you would use if you had a git repository for a website and you wanted to copy the files into /srv/http after the push
<Joerg-Neo900>
not to /srv/http but to another repo, with filtering out some files
<Joerg-Neo900>
since stupid git has ZILCH "file permissions" or access control on a per-file level
<atk>
there's also post-update and a few others, but if you just want to run a hook after a push finishes, post-receive will probably be good enough
<atk>
you can implement access control on a per-file level in pre-receive
<Joerg-Neo900>
the idea is to copy the commit incl author, comment and all, just replace some files by dummy content if they have a certain extension
<Joerg-Neo900>
wpwrak: ping!!!
<atk>
You must have pinged him to death.
<Joerg-Neo900>
atk: basically the idea is to achive a behavior similar to .htaccess so some files are not available via anonymous git clone/pull/fooo http://xxx.yy/zz
<atk>
oh right
<atk>
I see
<wpwrak>
what's the emergency ?
<Joerg-Neo900>
wpwrak: atk has valuable info to share
<wpwrak>
atk: so, tell one which horse to bet :)
<wpwrak>
s/one/on/
<atk>
The one with the weird name.
<Joerg-Neo900>
atk: (implement access control on a per-file level in pre-receive) would that be like "stdin tells the true original filename and you can decide whether it's allowed or not to add/delete/whatever this file" ?
<atk>
I'll have to investigate this proposal so give me a moment
<Joerg-Neo900>
of course, many many thanks! :-)
<wpwrak>
we want to selectively suppress files, but not reject the whole operation
<Joerg-Neo900>
actually I'd prefer to modify files to have dummy content instead
<Joerg-Neo900>
which in my book would vastly simplify the niche case handling
<Joerg-Neo900>
like "delete X" -- "BUT, there's no X -> ERROR"
<wpwrak>
this one now suppresses them cleanly. and since it's all git, i hope also "odd" things like symlinks (which don't have a "standard" patch equivalent) will work. but let's see.
<wpwrak>
we can do that. but then we need a script that redacts the diff, and also handles all the special cases. so this is a little more complicated.
<wpwrak>
;et
<wpwrak>
let's try the easy approach first
<Joerg-Neo900>
I think it's easier to simply sed the content, so no niche cases will occur
<Joerg-Neo900>
while with suppressing files you have a plethora of special cases like commits with no files but a commit message, like deletes of non-existing files, and whatbot else
<Joerg-Neo900>
plus the whole commit history will not get altered at all
<Joerg-Neo900>
with dummy files instead of the forbidden ones
<wpwrak>
so far, it handled everything i throw at it well
<wpwrak>
and which difference in the commit history do you dislike ? i already preserve what i can
<Joerg-Neo900>
what's wrong with dummy files? can't we get those? I think they serve our purpose better and in my book it should be simpler too
<atk>
I'm just going to mock up some python script
<Joerg-Neo900>
(( i already preserve what i can)) exactly my point. How would you preserve a commit that only creates a forbidden file?
<wpwrak>
i'd have to think about the interaction with other operations. what i have now is clean suppression. it's basically git talking to git. i don't even get in the way.
<wpwrak>
it's an empty commit
<Joerg-Neo900>
didn't think git would allow that
<Joerg-Neo900>
well, whatever works
<wpwrak>
what i do seem to lose is the commit-date and author-date duality. so the copy gets a false commit date. most things only care only abuot the author date anyway.
<wpwrak>
but then, few enough people would even know there is a difference ;-)
<Joerg-Neo900>
err
<Joerg-Neo900>
.oO(???)
<wpwrak>
(i didn't until i wrote eeshow and suddenly saw strange dates - because i used the commit date while i shuold have used the author date)
<wpwrak>
it's an efficient description. not short, though :)
<Joerg-Neo900>
what is the date?
<Joerg-Neo900>
time of push? tome of commit?
<wpwrak>
anyway, i have creation, change, deletion, symlinks work. now, rename and merge ...
<wpwrak>
it's a bit like mail header vs. mail envelope. but you really should read the above link if you want to understand the difference
<wpwrak>
we preserve the author date, which is what most people care about
<Joerg-Neo900>
and what happens to the other then?
<wpwrak>
there are a few choices. i think it just gets set to "now", i.e., the time the push happens
<Joerg-Neo900>
and no, I don't want to understand the difference, I only want to understand if and why it would be relevant to us
<Joerg-Neo900>
time of push is just fine
<Joerg-Neo900>
nobody cares
<wpwrak>
yeah. it makes the copy "imperfect", but that's the only practical implication i can think of
<Pali>
Joerg-Neo900: tiwlan chips needs for correct working model specific calibration data... where will they be stored in neo900?
<Pali>
e.g. in n900 they are stored in CAL filled in factory
<Joerg-Neo900>
Pali: there's no hw defined place for them to store. And the calibration can get done autonomously on device with a tool TI provides afaik
<Pali>
as those ti wl chips do not have own eeprom...
<Joerg-Neo900>
yes, I know
<Pali>
but do not if there is fully autonomous tool by TI
<Joerg-Neo900>
sorry?
<Pali>
for wl1271 is there some manual tool (you can specify params and that generate calibration binary file)
<Pali>
and where would be stored MAC address for wifi and bluetooth?
<Pali>
only on paper attached to neo900? :D
<Joerg-Neo900>
again, there is no definition in hw for that. It's up to the system
<Pali>
those are device specific data, so somewhere they need to be stored
<Pali>
either printed on device (or chip) or paper...
<Pali>
normally if you buy wifi card, it has stored MAC address in some eeprom and kernel driver will read it
<Joerg-Neo900>
I don't get it, do you ask for Neo900 UG to "throw dice" and provide a paper with the points printed on it?
<Joerg-Neo900>
well, when the hardware doesn't have any MAC stored to it then that doesn't make sense, no?
<Pali>
I'm asking how can somebody read MAC address for wifi chip which will be in neo900
<Joerg-Neo900>
o.I
<Pali>
normally each wifi chip must have assigned some MAC address from manufactor
<Joerg-Neo900>
o.O even
<Joerg-Neo900>
mist it?
<Pali>
so if you buy those TI chips you will receive also MAC addresses
<Joerg-Neo900>
if that's the case then we'll provide a calibration.bin file with the BSP
<Pali>
but somehow you need to distribute them to users
<Pali>
ok, so external file with BSP will contains those needed data
<Joerg-Neo900>
no, I don't think there ships any MAC with the TI chips
<Joerg-Neo900>
if there does then we will put it to a file and possibly also a sticker and attach alll that to the device
<Pali>
so neo900 would not have any eeprom for calibration and other device/hw supplied data?
<Joerg-Neo900>
actually I *think* it's "user" who provides the variable part of MAC to the tool TI provides. Or the tool calls RND() and prints out the result
<Joerg-Neo900>
no, not dedicated
<atk>
Joerg-Neo900: http://sprunge.us/FKUE - this isn't fully tested, and it doesn't work for a weird edge case I haven't worked out yet, but I've written a pre-receive hook which refuses pushes which try to change files in forbidden/
<Joerg-Neo900>
you can inplement CAL just like on N900 if you like
<Pali>
on N900 is there a big fuckup with wifi mac address
<Pali>
as it is stored in CAL and linux kernel obviously cannot read it
<atk>
But if you're willing to experiment a bit and make it more bulletproof I imagine it would be possible to fix the "git mv" case
<Pali>
so after each (re)boot kernel using random MAC address
<Joerg-Neo900>
atk: thanks a ton! wpwrak: could you look into it please?
<Pali>
and this break everything...
<Pali>
(MAC address filtering on router, DHCP assignement, ...)
<atk>
actually git mv into the directory works fine, git mv out of the directory is not allowed
<Joerg-Neo900>
Pali: but how is any of that a HARDware issue when you can set the MAC to anything you like?
<atk>
Probably on the internet someone has a ready made solution for this
<Pali>
Joerg-Neo900: it is hardware issue as hardware does not have (dedicated) eeprom with stored MAC address and calibration data
<Pali>
normaly in read-only mode
<Joerg-Neo900>
Pali: isn't it a question where and how kernel/system likes to store the MAC and similar config data?
<Pali>
and in simple format!
<Pali>
no as linux kernel expects that hardware itself (e.g. wl chip) provides MAC address
<Pali>
but for wl1251 it is not truth
<Pali>
it does not provide it
<Joerg-Neo900>
ok, to make you happy: I hereby define last 4 blocks of NAND as config storage, up to implementation to find a nice "filesystem" and data syntax for such config data
<Pali>
some wl1251 based devices fixed it by having dedicated eeprom where is that MAC address stored (at fixed offset)
<wpwrak>
atk: so this only lets the "good" files through, but not the "bad" ones ? even if a commit contains both ? or at that level does this operate ?
<Pali>
nokia decided to create fucking CAL format and store that wifi mac address; no fixed offsets; complicated CAL format;...
<Joerg-Neo900>
see above: this is "eeprom" and fixed offset
<atk>
It kills the push if it contains pushes to the forbidden files
<wpwrak>
atk: but such a push is perfectly legal
<atk>
It would be possible to write a client side hook to remove such files from the commit
<atk>
automatically
<atk>
so the conflict is never there
<wpwrak>
atk: the purpose is to filter files, not prevent them from being written to
<atk>
(along with the server side script)
<Pali>
Joerg-Neo900: if there will be definition that at offset 0xABC is MAC address and at offset 0xDEF are calibration data (with length X) it is OK
<Joerg-Neo900>
you're free to define that
<atk>
the thing with making changes like that is that it affects the whole integrity of the change, the server can't make that change because every time you push the server will get de-synchronised
<Joerg-Neo900>
I'll adopt whatever you come up with
<Pali>
not fully ideal, but maybe the best what we can do for such hw (tiwl) without own storage
<atk>
wpwrak: yeah, that would work, but you need two repositories for that
<Joerg-Neo900>
I'd *think* it's already in wl1251-nvs.bin, but then I might be mistaken on that
<atk>
wpwrak: for this you'll want the post-receive hook
<wpwrak>
atk: yup, that's the idea: one in which we work and one where we publish things
<atk>
wpwrak: then parse the input and just take the lines and put a .. between them, pass those to format-patch
<Joerg-Neo900>
Pali: it's probbaly just the problem with maemo flasher replacing that wl1251-nvs.bin with a non-unique one. So they needed to invent CAL to protect stuff in wl1251-nvs.bin from getting messed up by flashing, and overriding it by the MAC found in CAL
<wpwrak>
atk: is there some repo-side post-commit hook that would also tell me which branch the commit was to ? or something similar ?
<Pali>
/lib/firmware should not contains device specific files
<Joerg-Neo900>
should not? why?
<Joerg-Neo900>
FHS?
<wpwrak>
i.e., to use my "cg" script, i just need the commit hash, and i have to decide whether i care about the commit in question
<Pali>
you should be able to have one rootfs for more devices
<Pali>
FHS too
<atk>
wpwrak: do you need to know which branch?
<Joerg-Neo900>
well that's FHS :-)
<Pali>
and normally linux distributions reinstall stuff in /lib by package managers
<Joerg-Neo900>
then make that a symlink to /etc/firmware ;-)
<wpwrak>
atk: that's actually tricky :) right, better if i don't
<wpwrak>
that is .. i still have to tell git am .. somehow ....
<atk>
but basically that with your script should work maybe
<atk>
(post-receive hook)
<Joerg-Neo900>
Pali: is that ok for you? /lib/firmware/wl1251-nvs.bin -> /etc/firmware/wl1251-nvs.bin
<Joerg-Neo900>
assuming that wl1251-nvs.bin actually contains a MAC
<Pali>
wl1251-nvs.bin comes from linux-firmware package, so file /lib/firmware/wl1251-nvs.bin will be always overweritten by package manager (apt/zypper/yum)
<Pali>
thats fuckup
<Joerg-Neo900>
and if you feel ultra-conservative: /etc/firmware/wl1251-nvs.bin -> /dev/mtd2
<Pali>
but for n900 I have finally solution
<Pali>
userspace helper for loading firmware
<Pali>
that is still supported by kernel
<Pali>
(just systemd version of udev removed it)
<atk>
wpwrak: but you're quite right, if you start working with branches it gets a bit tricky
<Pali>
but we can reuse that design for n900 and fill kernel with adhoc generated nvs binary (which will be read from CAL)
<Joerg-Neo900>
how about replacing /usr/lib/libcal.so.1 in maemo with an implementation that reads stuff from a plain textfile on a well defined pathname?
<atk>
wpwrak: ah, yes, that's right, the "ref" you see in that argument is something like: refs/heads/master
<Pali>
probably hard to implement correctly ... (to not break dsme and other tools which use libcal.so.1)
<Pali>
but if neo900 will have stored firmware at fixed offset and mac address too (in nand), then whole firmware procedure can be implemented in kernel
<atk>
wpwrak: so you can work out the branch that way
<atk>
wpwrak: when a new branch is made, "old" will be 40 zeroes
<Joerg-Neo900>
Pali: you're free to do whatever you like on Neo900 :-)
<Joerg-Neo900>
and I'm even going to the extent to run a fab tool to set any file or storage location to whatever content you need them, when you provide such tool
<atk>
wpwrak: deletion of remote branches means that "new" is all zeroes
<atk>
for post-receive, you can see what the inputs are if you just symlink it to /bin/cat
<atk>
(post receive and pre receive both take input via stdin, update takes args)
<atk>
(but update gets ran once per what's effectively a line of text you get in the other two)
<Joerg-Neo900>
that's what I thought to be very convenient particularly for filtering by filename
<Joerg-Neo900>
but then, sonewhere that filtering needs to take effect. I assume the output of the hook will not replace the file content?
<atk>
No, that would be very messy
<atk>
The solution wpwrak is using - two repos - is much cleaner.
<atk>
But it might require some work.
<Joerg-Neo900>
will it reject a single file depending on output and/or return code?
<Joerg-Neo900>
err well, the two repos is exactly what I suggested to start with
<atk>
The solution works based on git apply --exclude
<Joerg-Neo900>
mhm
<Joerg-Neo900>
I'm out. No idea
<atk>
basically between the repositories the changes get passed as patch files
<atk>
then you just need to deal with branch creation / removal
<Joerg-Neo900>
we simply define there must nit be any branches in target repo
<Joerg-Neo900>
after all the target is our canonical http:// webgit
<Joerg-Neo900>
we don't need any branches there
<atk>
Yes, so refs can just be filtered if they're not the one repo you care about
<atk>
that should work...
<atk>
Only changes can be forwarded conditionally if it's the repo you care about
<Joerg-Neo900>
we have a ee repo with public access and no *.XY files, and we have an ee-internal with whatever files you can think of but no public access
<Joerg-Neo900>
I don't see a need to propagate branches from ee-internal to webgit ee
louis_ has quit [Ping timeout: 256 seconds]
louisdk has quit [Ping timeout: 260 seconds]
<how900>
atk: I'd be happy to benefit from your expertise
<atk>
I'm no expert :P
<wpwrak>
alas, we can't just ignore branches, since a file may get created in a branch (even if by accident), and then get merged into master.
<Joerg-Neo900>
I guess git is pretty much like c++ in that regard: only 3 persons on this globe *really* master it
<atk>
wpwrak: I'm not sure but that might still work sort of maybe if you ignore the branches, actually, I would have to test it.
<Joerg-Neo900>
ooh, so it needs to exist to get merged?
<Joerg-Neo900>
sucks
<atk>
The patches will still be there in the format patch
<atk>
I'd have to set up the full deal locally and test it but I think the patches will still get correctly generated with the changes from format patch
<atk>
because merges do generate a change that post-receive will see
<Joerg-Neo900>
that's what I assumed
<atk>
in fact, I can sort of test this now
<wpwrak>
Joerg-Neo900: well, that's a bit tricky, too. sometimes yes, sometimes not, it seems. e.g., i created a merge nothing odd came out of that. but we have a merge in the real "ee" that does result in the same patch twice if using git rev-list. so i need to puzzle out what's going on there.
<atk>
wpwrak: yes, you should be fine, I just tested it.
<Joerg-Neo900>
argh
<atk>
wait, where would you get the second patch in your "real" repo?
<atk>
you never touch it other than to format-patch (I hope)
mzki has joined #neo900
<wpwrak>
atk: d6ad7e9e46ece4ec0da8cbd9e2dbd7e80802c673 vs. fc7909b5c84d66f55530b0807a0d5bcdf52c9c8a is my problem. git format-patch -1 d6ad... reproduces fc79...
<wpwrak>
which git am understandably is unhappy about
<atk>
wait
<atk>
so I'll explain how I see it
<atk>
you have my script (that short 5 line thing) and all you do in there is add an if to check that ref == 'refs/heads/master'
<wpwrak>
atk: i pick commits individually (using git format-patch). the list of commits comes from git rev-list. but taht seems problematic.
<atk>
and only then do you do the git format-patch old..new and git am on master on the new one
<atk>
ah, don't rev-list, use the refs from post-receive
<atk>
and .. them
<wpwrak>
i know that i could just let git format-patch give me everything, which then would probably be consistent, but ....
<wpwrak>
... then i'd have trouble doing the incremental updates for pushes
<wpwrak>
at the moment i'm still at the initial "clone"
<atk>
if you give am a directory it will apply in order
<atk>
I think that's how I understood what you mean by "incremental"
<atk>
The only issue I can see is if someone does a --force push
<atk>
But that *might* just work out on the public repo, it just won't see the force push.
<wpwrak>
then it's rebuild time, no matter what :)
* Joerg-Neo900
fetches some popcorn
<atk>
you probably want that for the public repo because then you don't need to tell everyone to do a force pull
<atk>
they won't get affected
<atk>
or... hmm
<atk>
well, yes, for safety, rebuild at that point
<atk>
but for the automation, if you use the ref updates in post-receive you should be fine
<atk>
(just filter on which ref is being updated, only pass it on to the script if it's against master)
<wpwrak>
let's see if there are surprises if i use from ... to
<Joerg-Neo900>
Pali: mount -o bind /etc/CAL /dev/mtd1 ;-)
<Joerg-Neo900>
Pali: when doing this, you should do it early in preinit
<Pali>
no, CAL should be shred :D
<Joerg-Neo900>
hehehe ack
jonsger has quit [Ping timeout: 260 seconds]
<Joerg-Neo900>
well, seems even openwrt has it
<Joerg-Neo900>
iirc
<Joerg-Neo900>
and actually it's pretty silly simplistic a "filesystem", I just wonder what happens when it's full of garbage and needs "garbage collection" aka "defrag"
<Pali>
freemangordon already decoded what is nokia's cal implementation doing (when writing)
<Joerg-Neo900>
basically it's just a linked list of named records, with rev number, and when you write a new record it gets appended with revnumber = MAX(all_existing_rev_of_same_name)
<atk>
I haven't had adobe's crash player installed for a long time
<wpwrak>
Joerg-Neo900: hmm, old "ee" could be re-populated and made accessible (or maybe just unhidden, depending on how900 made it disappear), or atk could get access to the current hidden thing
<Joerg-Neo900>
atk: I need it
<Joerg-Neo900>
wpwrak: the latter
<Joerg-Neo900>
could you do that?
<wpwrak>
not sure about the rituals
<wpwrak>
how900: still around ?
<Joerg-Neo900>
I'd assume append pubkey to git's .ssh?
<wpwrak>
first, i need to see if i have access to that new repo myself ...
<Joerg-Neo900>
atk: could you please pastebin your pubkey?
<atk>
one moment
<Joerg-Neo900>
/query
<Joerg-Neo900>
but to a pastebin
<how900>
wpwrak: old ee is ee-full.git
<how900>
wpwrak: all conf goes through gitolite-admin.git (you have such access)
<Joerg-Neo900>
ooh
<Joerg-Neo900>
I already did brute force cat >>authorized_jeys
<Joerg-Neo900>
how900: could you please check what mess I created?
<how900>
Joerg-Neo900: sorry not now, I'm in bed :|
<Joerg-Neo900>
np
<how900>
but definitely not the way to talk to gitolite
<Joerg-Neo900>
should work
<how900>
Joerg-Neo900: gitolite add command before the key
<Joerg-Neo900>
unless gitolite has more user specific stuff stored elsewhere
<Joerg-Neo900>
I did too :-)
<how900>
ack
<how900>
well, it runs some post-processing stuff, maybe adding to each repository
<wpwrak>
Joerg-Neo900: just keep a few goats and a sharp knife ready, just in case extended measures are required. and maybe some coal, for drawing pentagrams, circles, and such on the floor :)
<atk>
on an unrelated note - my dmarc report aggregator works!
<Joerg-Neo900>
could somebody please post atk the correct git commands (incl port number!!) so he can test if it works?
tsuggs has joined #neo900
<Joerg-Neo900>
in a /query please!
l_bratch has joined #neo900
<Joerg-Neo900>
ok folks, this is sorted too (IOW works), and I need to go afk now
<Joerg-Neo900>
((<wpwrak> i would say "see for yourself". but of course, you can't ... :()) now he can
<atk>
wpwrak: doing a format-patch will flatten the commits as you see it happening
<atk>
one moment, I'll recreate a test
<wpwrak>
atk: just look at the commits i mentioned above (you can get the flattened monster in all its glory with git format-patch --root)
<atk>
Yes, I can see the specific commits
<wpwrak>
atk: when yuo see things up bad around the "xxx" aka d0639da848bbedc97aadb1bab53da1c3700623d6
<wpwrak>
s/up/go/
<wpwrak>
well, lemme try a more explicit sequence ...
<wpwrak>
then search for d0639da848bbedc97aadb1bab53da1c3700623d6
<wpwrak>
compare with what gitk (or similar) shows. doesn't look right, does it ?
<atk>
I can recreate the issue
<wpwrak>
perfect
<atk>
hmm
<wpwrak>
i wonder what the solution would be. it basically looks like a non-linearizable problem (at least not trivially). i guess merge does some n--way merge with A, B, and the merge commit (and maybe more)
<atk>
one moment
<wpwrak>
well, actually it doesn't have to
<wpwrak>
it doesn't need to know how to fix it since it just has the complete files
<wpwrak>
i'm getting tired :)
<wpwrak>
plan B would be to get some list, then just make "new" patches with git diff from step to step. they would normally be just like the commits, but sometimes they could cheat a little
<atk>
You were quite right, branches would need to be tracked to get it all working correctly.
<atk>
At least... probably.
<atk>
I might test on a local bare repository tomorrow.
<wpwrak>
plan C: pull out the commits, redact the trees and blobs, synthesize new trees plus a subset of blobs (i.e., minus the censored ones), build a map between original and redacted hashes, then piece together the commit graph from that :)
<atk>
I'm not sure if this issue is only caused by the fact that we're trying to use things from the past
<atk>
does it need to be a graph?
<wpwrak>
if we can't even handle the past, what chance do we stand against the future ? ;-)
<Joerg-Neo900>
does it break things or will it recover?
<Joerg-Neo900>
as in "we are not interested in a particular detail of history messed up in webgit"
<wpwrak>
well, the same pattern can occur again, especially with multiple people making changes
<Joerg-Neo900>
still doesn't really matter
<wpwrak>
but one can even trip over one's own feet and cause a merge. already that could break it.
<atk>
wpwrak: Well, the question is will the same pattern have the same issue if the steps are broken up as with how post-receive sees changes
<Joerg-Neo900>
does it break things?
<atk>
I'll try two tests tomorrow
<atk>
Joerg-Neo900: yes
<atk>
at least, in a certain scenario we're using
<Joerg-Neo900>
how?
<Joerg-Neo900>
segfault?
<atk>
but not because patches are in the wrong order, simply because merges can't be flattened that easily
<atk>
no, patches don't apply
<Joerg-Neo900>
modern art instead of schematics?
<atk>
no, git just refuses to take the patch
<Joerg-Neo900>
:-/
<atk>
but modern art would be good too, could sell that
<wpwrak>
it refuses to accept changes beyond the point of failure
<atk>
But I'll see tomorrow what it would take to recreate the branches and all that
<Joerg-Neo900>
wpwrak: how much effort is it to handle branches/merges then?
<wpwrak>
Joerg-Neo900: that's a very good question :)
<atk>
Hmm.
<Joerg-Neo900>
please always try the simlest least demanding solution
<atk>
You know, this wouldn't even be hard to test without anything fancy
<Joerg-Neo900>
stuff on webgit doesn't have to be perfect, just work particularly to pull HEAD(?)
<wpwrak>
atk: if it just follows all paths/branches, then it should be enough to change the same file in both paths/branches to get the problem
<atk>
I'm testing something now
<Joerg-Neo900>
would be nice but isn't really mandatory to also have all history for eeshow on public ee repo
<wpwrak>
what i'm a bit surprised about is that i can't find any mention of issues for git format-patch vs. merges. sure, most people use format-patch in a different type of workflow (i.e., with deliberate rebasing, very different from what we want here), but still
<Joerg-Neo900>
I'm feeling alike with the more generic problem of not exposing particular files
<Joerg-Neo900>
you'd think that should be fairly common a requirement
<wpwrak>
plan D would be to teach kicad to just honor symlinks. then we don't need to redact anything :)
<Joerg-Neo900>
heck it's even POSIX for regular files
<Joerg-Neo900>
"teach kicad"? :-o nope! no way
<Joerg-Neo900>
razher we use bindmounts
<wpwrak>
well, symlinks and file replacement have always been tricky. the open, truncate, then rewrite approach has the ugly properly of not being atomic anywhere
<wpwrak>
(teach) or lure someone on #kicad into doing it. got some tasty-looking bait ? ;)
<Joerg-Neo900>
no way. won't fly. I won't build nightlies
<atk>
wpwrak: alright, I just discovered that tracking the branches isn't going to solve it
<Joerg-Neo900>
neither will any of our subcontractors and volunteers
<atk>
I think the best solution will be that git log -p --pretty=email --stat -m --first-parent --reverse old..new
<atk>
that information isn't there in the post-receive
<Joerg-Neo900>
before we go down that path to doom of needing a project specific patched kicad, I rather nuke the whole public git stuff and revert to publishing plain download dirs
<Joerg-Neo900>
guys, you're awesome. This sounds excellent
<wpwrak>
we don't care about what goes on between the two ends anyway
<wpwrak>
whee, it passed !
<atk>
and I basically made my post-receive hook a symlink to /bin/cat so I could see what data was being passed in
<atk>
I ignored any changes to the other branch and only used the sha ids from the master branch
<atk>
and it generated the resulting lhs branch
<atk>
"fries" and "ketchup" commits from the "branch" branch disappeared, they ended up in the commit called "Merge branch 'branch'"
<wpwrak>
i crushes old..new, too (for the whole history). that gave be some mild bickering, but worked nevertheless
<atk>
it complained about training whitespace probably?
<atk>
trailing*
<wpwrak>
naw, i already have --whitespace=nowarn
<atk>
Well, as long as it's working that means I can go to sleep.
<Joerg-Neo900>
atk: many many thanks!
<atk>
It's no problem.
<atk>
e: see! I can be on topic sometimes.
<Joerg-Neo900>
lol
<wpwrak>
atk: great. thanks a lot !
<wpwrak>
the magic's in the -m (-m for "magic" :)
<atk>
Indeed
<atk>
Unfortunately format-patch doesn't have an equivalent for some reason
<atk>
But in this case, git log seems to work better by default (prints to stdout instead of files)
<wpwrak>
yeah. and a much more familiar command, too. at least the basics :)
<Joerg-Neo900>
now...:
<Joerg-Neo900>
apropos gitolite
<Joerg-Neo900>
gitolite: nothing appropriate
<Joerg-Neo900>
what the hell!? /srv/git/repositories/gitolite-admin.git
<wpwrak>
hmm, and we should probably make the commit date not "now" but track the author date. that way, recreating a repo from scratch should yield identical hashes. let's see if this works also beyond pure theory ...
<wpwrak>
yeah ! so we now recreate a perfect copy of the "censored" repo
<wpwrak>
Joerg-Neo900: diff -ru -x .git /home/n9/ee-full/ ee/ -> Only in /home/n9/ee-full/cvt: GTA04b7.kicad_pcb Only in /home/n9/ee-full/hw: neo900.kicad_pcb :)
Pali has quit [Remote host closed the connection]
<Joerg-Neo900>
wpwrak: awesome! excellent job!
<wpwrak>
now we just have to solve the incremental updates (i.e., the "push hook"). let's hope they're as easy as atk said. but that's something for tomorrow ...
<Joerg-Neo900>
could you look into gitlote repo and see what can get done to enable webgit already?
<wpwrak>
http://neo900.org/git seems to work here ? but in any case, your gitolite expert is hellekin. all i know about that critter is the name, and even that only barely :)
<wpwrak>
yes, but what's the problem with gitolite/webgit ?
<wpwrak>
or is there something in the repo that's now being shown ? i'm confused
<Joerg-Neo900>
well, I assumed that it's blocked just like you said a half a day ago
<wpwrak>
no, i think hellekin just re-initialized the repo
<wpwrak>
or maybe renamed and re-created
<Joerg-Neo900>
then what's diff -ru -x .git /home/n9/ee-full/ ee/
<wpwrak>
that was a local check that the local "clone" i made differed correctly from ee-full
<Joerg-Neo900>
aah, so can't we apply that to server already?
<wpwrak>
we, i can try to push that clone if you're in a hurry. but it will have me as the committer, not whatever eventually gets picked on the server
<Joerg-Neo900>
errr
<wpwrak>
so when that gets re-created, it will differ. while something created with the proper process (which isn't complete yet), shouldn't
<wpwrak>
committer != author. same difference as with the date
<Joerg-Neo900>
sorry, I can't follow
<Joerg-Neo900>
I thought there was a job now which could run on the server
<wpwrak>
lemme ask it like this: does ttp://neo900.org/git/ee/ _NEED_ to show content now ? or can that wait a day ?
<Joerg-Neo900>
no need
<Joerg-Neo900>
just a nice to have after one week of webgit down/dead
<wpwrak>
no, there's a big part of that job now. what we have now is "cloning" (and censoring) the repo. what we don't have yet is incorporating of updates
<Joerg-Neo900>
isn't that independent anyway?
<wpwrak>
plus a few minor details, like defining what we call the committer
<wpwrak>
it should overlap with what we've solved today
<wpwrak>
maybe the overlap is very large. maybe not. i don't know yet.
<Joerg-Neo900>
sorry, I'm completely lost
<wpwrak>
again, we have a process that can move everything from A to B