antocuni changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | "PyPy: the Gradual Reduction of Magic (tm)"
dddddd has quit [Remote host closed the connection]
forgottenone has quit [Read error: Connection reset by peer]
forgottenone has joined #pypy
<arigato>
solarjoe4: hi
<solarjoe4>
arigato, hi
<arigato>
I think so, if it is implemented in numpy with enough care: for example, on 32-bit linux "double" alignment is 4 bytes, but on 32-bit windows it is 8 bytes, I think
<arigato>
that's what C code and cffi do
<arigato>
but I would hope that the implementation in numpy, like cffi's, is based on really asking the compiler that compiles numpy or cffi
<arigato>
"""
<arigato>
Note that although almost all modern C compilers pad in this way by default, padding in C structs is C-implementation-dependent so this memory layout is not guaranteed to exactly match that of a corresponding struct in a C program. Some work may be needed, either on the numpy side or the C side, to obtain exact correspondence."""
<arigato>
ah, it seems to be based in numpy on 'offsetof(struct {char c; type v;}, v)', which is exactly like cffi
<solarjoe4>
good thought, I will forward this question to the numpy mailing list, I just hope this works fine, because it might be very convenien
<arigato>
if it's really based on this, then yes, it should work fine
<solarjoe4>
where you got that from? :)
<arigato>
by following the link to " PyArray_Descr.alignment"
<mattip>
I could not find a way around the double-define of the dt to deal with padding needed to pack an array of c structures
<mattip>
which is a different issue than packing within the c strucure itself
kanaka has quit [Ping timeout: 248 seconds]
<solarjoe4>
mattip: did you try the keyword align=True in the dtypy constructor?
<arigato>
ah, you can give explicit offsets to numpy
jamesaxl has quit [Read error: Connection reset by peer]
jamesaxl has joined #pypy
<mattip>
solarjoe4: now I did, and indeed that corrects the padding
<solarjoe4>
arigato, mattip: now we just need a small function to create the dtype out of the definition in cffi using cffi.typeof, but that should be possible :)
<arigato>
for example, makes a dtype with an extra _everything that spans the whole structure, and you can give its size, here 35
<mattip>
+1
<solarjoe4>
mattip, oh you are right, i mixed that up
<mattip>
s/I get/I get an error/
<mattip>
arigato: how do I get the type of a field of a struct in cffi?
<mattip>
there used to be some incantation, maybe via ffi.lib() ?
<arigato>
ffi.typeof("mystruct").fields
<arigato>
ffi.typeof("foo").fields[0][1].type
<mattip>
thanks
<mattip>
is the list from ...fields always ordered by the layout in memory?
<mattip>
so you could iterate that to create the dtype, and add a np.void at the end
<arigato>
it is in the definition order
<mattip>
cool
antocuni has quit [Ping timeout: 264 seconds]
<arigato>
numpy and cffi are still kind of strangers to each other, enough that neither of them feels that it should natively provide the logic to deal with that
<arigato>
if you implement it in a general enough way as a small reusable library somewhere, please tell both us and numpy :-) at least we can link to it from the cffi docs
<solarjoe4>
for my code, I guess I will try to solve it by dynamically creating the dtype using ffi.typeof("foo").fields and then check the size and offset of the fields with ffi.offsetof and the numpy.dtype.fields
<solarjoe4>
of course using the align=True :)
<arigato>
sounds like a good plan
<mattip>
hint - you can use ctype to get the canonical name
<mattip>
[str(x[1].type.cname) for x in ffi.typeof('mystruct').fields]
<mattip>
['int', 'int', 'float', 'double']
<mattip>
or even [x[1].type.cname for x in ffi.typeof('mystruct').fields]
<solarjoe4>
but what to do with something like <ctype 'float3[3]'> ?
<solarjoe4>
how could that translate?
<solarjoe4>
it should be a shape (3, 3) array at best
<mattip>
you would need a map of ctype -> numpy dtype descr
<solarjoe4>
check with ffi.typeof("foo").fields.kind for "array"
<solarjoe4>
?
<mattip>
yes, then recurse to get a dtype
<mattip>
once you get the recursive value, add it to the map
<solarjoe4>
puh, damn hard to get convenient dynamic code :)
demonimin has quit [Ping timeout: 268 seconds]
inad922 has quit [Quit: Leaving]
demonimin has joined #pypy
demonimin has joined #pypy
marky1991 has joined #pypy
marky1991 has quit [Ping timeout: 240 seconds]
marky1991 has joined #pypy
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
<mattip>
thinking about third-party dependencies
<mattip>
cpython has a source-repo and a binary-repo, each branch of the binary is by external library
<mattip>
I am thinking of doing the same, each or test run would start by updating the dependencies
<mattip>
but if this is inside the pypy repo, it would be erased each time by hg purge --all
<mattip>
should it be a subrepo, or another directory next to pypy
<mattip>
on windows, we would then modify the INCLUDE, PATH, and LIB env variable appropriately
<mattip>
so if it is outside pypy and by chance the user has a directory there with the wrong name, we would pick it up
<mattip>
ok, it seems that hg cloning a repo inside pypy works, and even without declaring that a subrepo it is ignored by hg status or hg purge
tayfun26 has joined #pypy
<mattip>
so my strategy will be - if [ ! -d externals ]; do hg clone externals; done; cd externals; hg pull; hg update <branch>;
<mattip>
where branch will be vs9 or vs14
<mattip>
all that on win32
<mattip>
hmm, then branch should be vs9_32 or vs14_32 for now
forgottenone has quit [Ping timeout: 240 seconds]
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
antocuni has joined #pypy
<arigato>
er, are you sure about "hg purge --all"?
<arigato>
not just "hg purge"
<mattip>
in builds.py, command="hg --config extensions.purge= purge --all"
dddddd has quit [Ping timeout: 240 seconds]
<arigato>
yes, I'm saying, I would expect "hg purge --all" to remove unrelated repositories that have been checked out inside
<mattip>
tested by hg clone buildbot inside pypy, and hg purge --all at the level of pypy, buildbot was not removed
<mattip>
huh? why is benchmark failing? timeout when running own/bm_mdp.py ?
marky1991 has quit [Ping timeout: 276 seconds]
inad922 has quit [Quit: Leaving]
realitix has quit [Read error: Connection reset by peer]
Rhy0lite has joined #pypy
realitix has joined #pypy
forgottenone has joined #pypy
adamholmberg has joined #pypy
solarjoe4 has quit [Quit: Leaving]
kanaka has quit [Changing host]
kanaka has joined #pypy
Joghurt has joined #pypy
<Joghurt>
Since yesterday, rpython.readthedocs.io is rather... empty. Is this intentional?
yuyichao_ has quit [Ping timeout: 256 seconds]
marky1991 has joined #pypy
inad922 has joined #pypy
<arigato>
Joghurt: thanks!
<arigato>
seems that readthedocs failed to connect to bitbucket, but that left it in an empty state
<arigato>
I just triggered a build again. might take a while, if the repository was removed by the failure (which I don't know)
<Joghurt>
ty
nopf has joined #pypy
yuyichao_ has joined #pypy
realitix has quit [Read error: Connection reset by peer]
realitix has joined #pypy
<mattip>
rpython docs are back, look ok
<arigato>
ah. but that was achieved by randomly clicking around and I fear it would have stopped working again. I finally wiped the whole repo clone (found out how) and now it's cloning again
<arigato>
ah no, it was finished a few minutes ago, and it still looks good. good
forgottenone has quit [Ping timeout: 240 seconds]
<mattip>
bye
mattip has left #pypy ["Leaving"]
antocuni has quit [Read error: Connection reset by peer]
antocuni has joined #pypy
tayfun26 has quit [Quit: tayfun26]
yuyichao_ has quit [Ping timeout: 264 seconds]
Joghurt has quit [Ping timeout: 240 seconds]
realitix has quit [Remote host closed the connection]
forgottenone has joined #pypy
tbodt has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
oberstet has quit [Ping timeout: 240 seconds]
tbodt has joined #pypy
antocuni has quit [Ping timeout: 252 seconds]
yuyichao_ has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]