<openfpga-github>
[openfpga] rqou opened issue #69: gp4prog times out too quickly switching modes on Windows https://git.io/vyT3i
<openfpga-github>
[openfpga] rqou opened issue #70: gp4prog: Don't try to detach kernel driver on Windows https://git.io/vyTsv
<rqou>
so yeah, smoke testing on Windows actually works
<rqou>
no builds tonight though
<rqou>
need to fight with yosys on windows
<rqou>
and i need to readjust my sleep, so i'm going to bed
<rqou>
and amazingly, this is a cross-build
<cr1901_modern>
Detach kernel driver? What do you mean?
eduardo_ has joined ##openfpga
<rqou>
so libusb is supposed to allow accessing USB devices from userspace
<rqou>
but you can only do that if the kernel isn't trying to use the device at the same time
<cr1901_modern>
right, you need a special kernel driver to do it
<rqou>
only on Windows
<rqou>
so what happens is
<rqou>
on Linux, USB devices are accessible in /dev as files
<rqou>
some special command disables the kernel driver for a particular device, signalled through the /dev file
<cr1901_modern>
oh that's cool!
<rqou>
on Mac, IOKit somehow has something that exposes USB devices similar to Linux
<cr1901_modern>
On Windows I just uninstall the kernel driver and always access through libusbk.sys
<cr1901_modern>
erm, the default* kernel driver
<rqou>
but this will be overridden by more-specific drivers
<rqou>
so on Mac if an existing driver is claiming some device you want
<rqou>
you need a "codeless next" with even higher priority
eduardo has quit [Ping timeout: 240 seconds]
<rqou>
that then does nothing and lets the normal IOKit logic happen
<rqou>
on Windows you always need to load a specific driver
<cr1901_modern>
"codeless next"?
<rqou>
*kext
<rqou>
damn autocorrect
<rqou>
kernel extension
<cr1901_modern>
"on Windows you always need to load a specific driver" <-- this is presumably referring to libusbK.sys
<cr1901_modern>
I only load the driver once; when I'm installing the device
<rqou>
yeah, but in this case we can use the windows built-in HID driver
<rqou>
although iceprog is going to be fun
<rqou>
iceprog uses libftdi
<rqou>
so you need to replace the normal FTDI driver with libusb
<nats`>
Zadig
<cr1901_modern>
libftdi certainly does need libusb
<cr1901_modern>
This seems like a lot of code for little benefit. Installing a libusb driver isn't that big of a deal. Those on Windoze are aware the driver situation is screwy
<cr1901_modern>
(and yes, libwdi/Zadig has done a lot to alleviate the pain)
<cr1901_modern>
The guy who wrote libwdi has gone to great lengths to make distributing libusb compat code as painless as possible
<cr1901_modern>
rqou: Thanks for the info on how Linux does it. I've been wondering how it works for a while/never looked it up :P
<cr1901_modern>
Always great when someone else does it for me :D
<rqou>
although FTDI on Mac is even more awesome nowadays
<rqou>
Apple has a kext, but the FTDI blob userspace can't use it
<rqou>
the FTDI blob userspace uses the IOKit mechanism
<cr1901_modern>
Why is a "codeless kext" sufficient to get libusb to talk directly to the USB device?
<rqou>
so even the official proprietary blob requires some stupid kextunload
<rqou>
a "codeless kext" can match with a higher priority than the normal kernel driver
<rqou>
so the normal kernel driver doesn't load
<cr1901_modern>
right, but a "codeless kext" still is a driver for a device that takes up resources?
<rqou>
but because the new kext you added doesn't have any code
<rqou>
it falls back to the IOKit default handling
<cr1901_modern>
oh...
<rqou>
IOKit has some complicated logic for selecting drivers to load
<rqou>
governed by XML files :P
<cr1901_modern>
so libusb relies on that fallback for any of the data it wants to send to get through?
<rqou>
yeah
<cr1901_modern>
As for the windows situation, I think documenting/linking to Zadig is a better idea than using the HID to keep the source base clean.
<cr1901_modern>
A lot of boards using FTDI chips, and a lot of programmers use libftdi
<rqou>
er, the greenpak board uses HID
<rqou>
the Lattice boards use FTDI/Zadig/whatever
<cr1901_modern>
ahhh, so leave the lattice boards/iceprog alone, change Greenpak
<cr1901_modern>
that's prob fair. That's up to azonenberg of course. It's going to be less code on Windoze, and not require a special driver. But will it integrate well into the codebase (that's his call)?
<rqou>
er, what?
<rqou>
you just need to comment out one line to make gp4prog work
<rqou>
iceprog is a much bigger mess
<rqou>
I'll explain more tomorrow, going to sleep
<cr1901_modern>
"We actually want to use the Windows native HID driver because libusb knows how to talk to that." <-- oh... I did not know this
<cr1901_modern>
I also apparently didn't read
qu1j0t3 has quit [Ping timeout: 240 seconds]
qu1j0t3 has joined ##openfpga
ZipCPU|Laptop has quit [Quit: Leaving]
azonenberg_work has quit [Ping timeout: 240 seconds]
azonenberg_work has joined ##openfpga
kuldeep has quit [Remote host closed the connection]
<pie_>
ithink i still have a few vids from you i need to watch
<pie_>
lol, "paizoelectric"
Hootch has joined ##openfpga
m_w has joined ##openfpga
<openfpga-github>
[openfpga] azonenberg pushed 1 new commit to master: https://git.io/vyTwM
<openfpga-github>
openfpga/master 9f773c1 Andrew Zonenberg: Updated to latest logtools
<azonenberg>
cr1901_modern, rqou: keep in mind also that right now libjtaghal uses the blob libftd2xx
<azonenberg>
i had reliability issues with libftdi, dont remember details
<azonenberg>
If somebody wants to try porting my FTDIJtagInterface class to optionally use libftdi and test, i wont complain
<rqou>
right, but iceprog doesn't
<azonenberg>
Yeah what i mean is
<azonenberg>
whatever system we come up with has to play nicely
<rqou>
so now you need both
<azonenberg>
incidentally this is one of the reasons that i a) used a custom usb PID for my jtag stuff, to avoid loading the default usb-serial driver etc
<azonenberg>
and b) plan to deprecate FTDI usb-jtag for my projects in the long term
<azonenberg>
basically as soon as i have a working starshipraider v0.1 alpha
<azonenberg>
i'm going to purge my lab of ftdi :p
<azonenberg>
and never look back
<rqou>
ah you have a custom PID
<rqou>
that helps you but not iceprog :P
<rqou>
only Linux works sanely with having both libftdi and libftd2xx
<azonenberg>
Yes i use 0403:8028
<azonenberg>
and use ftd2xx on that
<azonenberg>
I also support the Digilent API using whatever they use under the hood
<azonenberg>
and my own Ethernet protocol
<azonenberg>
Down the road i plan to consider digilent/ftd2xx back ends deprecated and keep them for compat only
<azonenberg>
and have sockets to starshipraider or api-compatible boards be the primary back end
<rqou>
I wonder why nobody wrote a thing that wraps libftd2xx and pretends to be libftdi
<azonenberg>
or the other way around?
<azonenberg>
No idea
<azonenberg>
Right now i support the blob b/c it seems slightly less unreliable than the open tool
<azonenberg>
but both are terrible
<azonenberg>
the blob uses a massive amount of cpu per open handle... polling? not sure what else it could be doing
<azonenberg>
having like six ftdi devices plugged into a beaglebone pegged the cpu
<azonenberg>
without actaully sending any data
<rqou>
wow you actually tested on ARM?
<azonenberg>
Yes, the intent at the time was to have my beaglebone be the front-end / landing pad node for my fpga cluster
<azonenberg>
that would go push jobs out to them for testing
<azonenberg>
it just didnt have the horsepower
<azonenberg>
My next-gen fpga cluster, which has been a LONG time coming but is sloooowly progressing
<azonenberg>
will have a hardware accelerated management card that's basically a starshipraider minus the level shifters
<rqou>
ah I see, but you weren't using yosys on ARM
<azonenberg>
Yes, this was jtag only
<azonenberg>
the build was running on an x86 cluster
<rqou>
which works btw (icestorm flow)
<azonenberg>
Doesnt surprise me, but i didnt want to wait until next week to get a compile done
<azonenberg>
:p
<azonenberg>
The big bottlenecks i had were ISE and yosys SAT/SMT solvers
<azonenberg>
both of which would saturate many CPU cores without breaking a sweat
<rqou>
if you poke around my Jenkins you might notice something interesting about the ARM build slave
<azonenberg>
oh? qemu instance?
<rqou>
oh I wasn't doing verification, just synthesis
<rqou>
yeah it's qemu-user on an Intel processor :P
<rqou>
might be not any slower than an actual RasPi
cybernaut is now known as GenTooMan
<rqou>
the osx slave is *also* a qemu instance (qemu-kvm)
<rqou>
this is why the builds slow each other :P
<azonenberg>
lol i see
<azonenberg>
and yeah some of my current formal models will saturate an x86 core (yosys's default solver isnt parallelized) for 10+ minutes
<rqou>
also I moved builds off of my linode onto my NAS thing
<rqou>
because linode has no more RAM due to Minecraft :P
<azonenberg>
Thats a lot faster than the old SAT solver, SMT is way better
<azonenberg>
but still a cpu hog
<rqou>
hmm, is there a parallelized one?
<rqou>
I have 10 cores in my workstation
<azonenberg>
I dont know but for my use case that isnt super duper important
<azonenberg>
b/c i typically have many formal models
<azonenberg>
and if i'm running several plus synth, par, and fpga testing
<azonenberg>
i can easily saturate many cores even if i cant parallelize any one operation
<rqou>
you should build a Broadwell Xeon/HEDT build machine like me
<azonenberg>
So right now my desktop is a haswell i5 with 32GB of DDR3 and only one pcie x16 slot
<azonenberg>
meaning i cant fit a 10g NIC and a nice GPU at the same time
<rqou>
i5 -> not HEDT
<azonenberg>
My near-mid term plan, like over the summer
<azonenberg>
is to move that mobo and ram to a 2U case in the garage using an existing power supply and pxe boot it off my NAS
<rqou>
you need a "X99" motherboard (for a "consumer" version)
<azonenberg>
maybe stick a 10g nic in the empty slot since i wont need a gpu for the server version
<azonenberg>
then replace my desktop guts with a dual socket broadwell xeon with 64-128GB of DDR4
<rqou>
anyways, my unattended NAS+build machine is really slow
<rqou>
it's a Kaby Lake Pentium :P
<azonenberg>
When you have 32G of RAM and processes occasionally get oomkilled or fail to launch (like gimp working on large images)
<azonenberg>
thats a sign you probably need to upgrade
<azonenberg>
Anyway, that would leave me with more pcie slots on the desktop too
<azonenberg>
so i could put a gpu and a 10g nic in it
<cr1901_modern>
I don't see the need to change iceprog for Windows. It's going to be reimplementing a decent amount of code for little gain (not to mention I don't care for the blob)
<rqou>
wow you're going for an actual real server chipset instead of me cheaping out on an X99 :P
<rqou>
I'm running a dual-socket capable Xeon, but X99 can't support dual sockets AFAIK
<rqou>
this is an unholy consumer+server mashup
<rqou>
you can't do this anymore Skylake+
<azonenberg>
yeah i have parts picked out that will do it right
<azonenberg>
going ECC too
<balrog>
any thoughts on AMD Ryzen?
<azonenberg>
it would only cost me around $1K in guts
<balrog>
(apparently all of their CPUs support ECC)
<balrog>
I don't think they're doing dual socket yet but single socket 8C/16T for $499
<rqou>
on my NAS I'm running a strange opposite of a consumer CPU on a server chipset
<rqou>
apparently for strange product differentiation reasons you need the server chipset for ECC
<rqou>
but the low end Pentium also has ECC enabled
<rqou>
azonenberg: $1k in guts for a dual-socket Broadwell? how?
* azonenberg
goes to look at newegg wishlist
<azonenberg>
Newegg doesnt have the stuff in stock so i cant get a current price quote
<azonenberg>
two E5-2630 v4 xeons
<azonenberg>
but i had $271.99 for a supermicro MBD-X10SRA-O
<azonenberg>
and a 4x 16GB DDR4 2133 ECC ram kit from kingston
<azonenberg>
Some projects, like Splash, use much more posix-isms
<rqou>
is there just a delay that needs increasing or some other problem?
<azonenberg>
splash wont be running on non-posix any time soon
<rqou>
i haven't actually looked
<azonenberg>
i havent either
<azonenberg>
whitequark: want to look into that?
<azonenberg>
i didnt write most of gp4prog
<whitequark>
which part
<whitequark>
69?
<whitequark>
oh
<azonenberg>
i started the very basic skeleton, then whitequark picked it up from me and i made a few minor fixes since then
<whitequark>
azonenberg: my board is in another country
<rqou>
68 is fun too
<whitequark>
so thats troublesome
<azonenberg>
but the vast majority of my contribution to the project has been in the PAR as well as the yosys cell library
<whitequark>
I don't have an rpi2
<whitequark>
I have an rpi3
<rqou>
but 68 will be fun to try to debug :P
<whitequark>
on rpi3 it works every time
<rqou>
oh really?
<rqou>
wtf
<rqou>
using my binary?
<rqou>
maybe it's a rpi2 kernel bug, but i don't care enough to look into it
<whitequark>
I've built it myself
<whitequark>
long ago
<rqou>
natively or cross? i'm curious how long a native build takes
<whitequark>
natively
<whitequark>
uh half a hour or something?
<whitequark>
rpi3 isnt shit anymore
<whitequark>
its about on par with a mid-end smartphone
<rqou>
half an hour including yosys?
<whitequark>
yeah
pie_ has quit [Read error: Connection reset by peer]
<whitequark>
I think I installed abc from raspbian?
<rqou>
damn
<rqou>
ah ok
<rqou>
yosys+abc takes about two and a half hours for qemu to build
<rqou>
great, abc has mixed line endings
<lain>
isn't there a git setting to normalize on checkout?
<rqou>
i have no idea, and it doesn't matter because abc is in hg :P
<rqou>
in general i wouldn't use such settings because i don't want my source control system magically changing my files
<rqou>
heh this reminds me of a great environment that existed when I was at NI
<rqou>
they used perforce for source control
<rqou>
but they had two repos, not one
<rqou>
the reason was because one was configured with unix semantics (case sensitive, LF) and one was configured with windows semantics (case insensitive, CRLF)
<rqou>
i don't know why, but there seem to be various groups at $FANCY_SCHOOL that really like hg
<rqou>
(everybody else seems to use git)
<lain>
hg was git before git was git
<rqou>
sure
<rqou>
but hg's definitely losing the popularity contest right now
<rqou>
i think my best hg story was the one class where the TAs really preferred hg
<rqou>
but i was in a project group where everybody preferred git
<rqou>
so we all ended up using git-to-hg bridges to interact with the TA's repo
<mtp>
i don't hate perforce
<mtp>
which is a shock to a lot of people
<nats`>
hehe I pretty like it
<mtp>
(i hate git, prefer fossil)
<nats`>
and I fucking hate git and I prefere HG but I have to use git sadly
<rqou>
wow lots of "hate git" in here
<whitequark>
i like git
<whitequark>
i find it very straightforward personally
<rqou>
i dislike hg due to muscle memory not working
<rqou>
"it's like git but everything is renamed" :P
<whitequark>
other people has led me to believe my brain is warped because of that
<lain>
git has too much magic... I like it more than any others, but I still feel there's far too many ways to do a given thing, and it's too easy to get yourself in a situation where you have to invoke magical incantations from the internet
<lain>
like, for day to day stuff you rarely run into issues
<lain>
but sometimes something goes wrong somehow, maybe a script derped up or who knows, and it rapidly throws you out into the wilderness :P
<whitequark>
i once got screamed at for not hating git
<whitequark>
to quote, "tolerance of git is an unethical act"
<mtp>
whitequark, my problem with git is that you have to pickle yourself in the internals in order to use it effectively
<mtp>
like, it's close to impossible to operate it well without understanding how it works
<nats`>
an you can totally break a git repo from remote
<nats`>
I don't like that
<lain>
I once heard what I feel is a very accurate description of git: git is not a version control system, git is a build-your-own-VCS kit. it's got all the parts you need, they're just not setup for actual use yet ;)
<nats`>
and yes too many magical option to know before using it
<whitequark>
mtp: I think I agree with that
<whitequark>
it's probably a consequence of how it was written
<rqou>
whee, segfaulted collect2/mingw-w64
<rqou>
oh, apparently due to wrongness i tried to link an elf into a PE
<rqou>
yeah that doesn't work :P
<rqou>
alright, all the tools are hacked up enough to have windows binaries output
<rqou>
time to convert from "bash history" to something more manageable :P
<rqou>
after this i'm going to probably want to start upstreaming build system fixes