<antocuni>
maybe we should try to contact some of the PyPI admin. I don't know what is the policy for these cases, but having an user "pypy" which is unrelated to us doesn't look very good
<antocuni>
e.g., suppose he/she decides to release some package which is harmful
<antocuni>
dstufft: ^^^ do you know who is the right person to contact for this?
ronny has quit [Ping timeout: 240 seconds]
<mattip>
antocuni: fwiw, there is no user "numpy", the numpy PyPI listing shows a number of persons who are maintainers
* mattip
just learning how the whole thing is set up
<mattip>
so perhaps the request should be "user names that are also popular in the python lexicon should be rejected"
raynold has quit [Quit: Connection closed for inactivity]
<antocuni>
mattip: I guess that if you try to register the user "numpy", you will succeed
<Rotonen>
that'll be a delightful cat and mouse game to occupy all the volunteers with
<antocuni>
one simpler way would be to unify the namespaces of projects and users (unless it's already like that)
<antocuni>
so you cannot have a numpy user because there is already a numpy project
<antocuni>
although it would have not helped for pypy because there is no "pypy" project on pypi, of course
<antocuni>
(maybe we should register one, just in case?)
<Rotonen>
the current scheme allows for enough flexibility without extra overhead costs of human effort
ronny has joined #pypy
<Rotonen>
there's the silly cases of someone's alias is now a project name of a newer project so what to do then etc.
<mattip>
the current situation + inability to reach users (issue listed above) seems ripe for abuse, but
<mattip>
anyhow PyPI is known for a liberal upload policy for good and for bad
* mattip
bbl
mattip has left #pypy ["bye"]
gclawes has joined #pypy
oberstet has quit [Ping timeout: 255 seconds]
jcea has quit [Remote host closed the connection]
jcea has joined #pypy
nbjoerg has joined #pypy
<nbjoerg>
loooking at cffi issue 321, I am wondering what the purpose of malloc_closure.h is
<nbjoerg>
i.e. why is cffi not using the closure allocation from libffi?
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 255 seconds]
cstratak has quit [Quit: Leaving]
<LarstiQ>
nbjoerg: Armin can tell you why, my guess would be: either libffi being buggy, or it being slow
<nbjoerg>
well, it sucks having to reimplement proper mprotect aware closures over and over again...
<nbjoerg>
and for libffi I have done that already
cstratak has joined #pypy
arigato has joined #pypy
<LarstiQ>
nbjoerg: I suspect you need a pypy implementation anyway, in order to not make libffi a blackbox the jit can't reason about
<nbjoerg>
huh? the point of libffi is kind of exactly that
<nbjoerg>
and it's not like you can reason about random C functions anyway :)
jacob22_ has joined #pypy
Ryanar has joined #pypy
Ryanar has quit [Client Quit]
dash has quit [Ping timeout: 246 seconds]
oberstet has joined #pypy
antocuni has quit [Ping timeout: 246 seconds]
rokujyouhitoma has joined #pypy
vkirilichev has quit [Remote host closed the connection]
rokujyouhitoma has quit [Ping timeout: 248 seconds]
gclawes has quit [Ping timeout: 276 seconds]
graingert is now known as jreback
jreback is now known as graingert
<arigato>
nbjoerg: historically, cffi doesn't use malloc_closure.h because it was created at a time where this file didn't exist in all libffi versions out there
<arigato>
nbjoerg: but now, I explicitly don't use malloc_closure.h because I feel it opens a major security risk in some cases
<arigato>
anyway, we're talking about two pages of C code
<arigato>
...though I'm talking about issue #231, not #321
gclawes has joined #pypy
bendlas has quit [Ping timeout: 240 seconds]
<nbjoerg>
yes, same kind of issue
<LarstiQ>
nbjoerg: feel free to ignore me, just listen to arigato
<arigato>
:-)
<nbjoerg>
arigato: #321 is effectively the same problem as #231
<arigato>
ah, nice to know
<nbjoerg>
we have a working version of the ffi closure handling
<nbjoerg>
without going to /tmp
<arigato>
and in a fork-safe way?
<nbjoerg>
the branch has conflicts
<nbjoerg>
yes
<arigato>
cool
<nbjoerg>
combination of two things: allowing mmap to specify protections that mprotect can enable later and mremap flag that can duplicate a mapping
<arigato>
based on the branch 'ffi_closure_alloc' of cffi?
<arigato>
how do you use mremap to duplicate a mapping? from the docs (on linux) it seems to optionally move it, but not duplicate it
<nbjoerg>
can I get a raw diff from bitbucket between two commit ids?
<arigato>
I don't know, I can only tell you how to do that from the command line
realitix has quit [Ping timeout: 248 seconds]
<arigato>
you could try to integrate that into the cffi trunk, ignoring the 'ffi_closure_alloc' branch. I'd of course prefer something a bit more portable, but it's a step :-)
<nbjoerg>
as far as selinux is concerned: badger linux distros to ship a version with atfork hook...
rokujyouhitoma has quit [Ping timeout: 248 seconds]
<arigato>
pthread_atfork()? it exists
<arigato>
the problem is that it's not obvious how to fix the problem using only pthread_atfork()
<arigato>
(unless we're ok with never freeing and leaking any callback pages that are alive at any fork())
<arigato>
NetBSD has the advantage of supporting pages that are shared inside a process, but not across fork(), or so I think---if I'm wrong, then what you linked to above is again a major security bug in case of fork()
<nbjoerg>
they are not file-based
<nbjoerg>
it uses normal anon memory
<nbjoerg>
I think the only sane way to implement it on Linux is:
<nbjoerg>
(1) keep a linked list of all closures
<nbjoerg>
(2) on fork, create a new file
<nbjoerg>
(3) iterate the list of closures, copy them to the new file, re-mmap them
<arigato>
you don't need actual files, but on linux you need to use mmap(MAP_SHARED), not MAP_PRIVATE
<arigato>
because MAP_PRIVATE can never appear at more than one place
<nbjoerg>
that's what you need the file for
<arigato>
no, you can do MAP_SHARED | MAP_ANON
<nbjoerg>
it doesn't have to linked
<arigato>
but yes, I see the point: in pthread_atfork() you would manually un-share the pages
<arigato>
or, at least, unshare them from the parent
<arigato>
my copying into a fresh MAP_SHARED mmap
<arigato>
s/my/by
<nbjoerg>
(I'm not even sure what MAP_SHARED|MAP_ANON is supposed to mean)
<arigato>
it means a non-file-based mapping, but otherwise the page is marked MAP_SHARED, i.e. you can map it at several places and fork() won't duplicate it
<nbjoerg>
so non-cow for fork. seems stupid and broken, but I'm not surprised by that
<nbjoerg>
I bet systemd uses that :)
<arigato>
it feels like an assumption that was baked into linux's mm system long ago: each page of actual RAM can be either SHARED or PRIVATE, and PRIVATE means always cow and SHARED means never
<kenaan>
rlamy default bb398c8ed6e0 /pypy/module/cpyext/: Define PyMemoryViewObject from C code
<arigato>
(I think you can "duplicate" a PRIVATE mapping so that it appears twice in the process, but each copy is then cow)
<arigato>
(which would be fine, if libffi supported that, but it doesn't)
<nanonyme>
arigato, could it?
<nbjoerg>
I don't think linux offers an interface for that
<nbjoerg>
I added one in NetBSD exactly so that I can properly deal with different JIT mechanisms sanely...
<arigato>
nanonyme: yes, it could pre-fill the RW page with a sequence of stubs, and then make the RX copy of it; it needs to be done a bit carefully so that the stubs don't need to be tweaked later
<arigato>
nbjoerg: I think there is an interface but maybe not completely exposed to the user---thinking about fork() itself, and about the virtual machine monitor which re-shares pages
<arigato>
and of course details like a large mmap() which gives a single null page, cow
vkirilichev has quit [Remote host closed the connection]
vkirilichev has joined #pypy
jcea has joined #pypy
rokujyouhitoma has joined #pypy
<njs>
I don't understand how having a page mapped as W in one place and X in another place is any different security-wise than having a single page mapped W+X
<simpson>
Ooh, I know this one!
<simpson>
It's because of rights attenuation.
<simpson>
Assuming that you can't forge references, it's the same kind of division as having a read-write capability and read-only capability for the same resource. Which capability you're handed changes which rights you have.
<njs>
okay, but... pointers are not capabilities :-)
<dash>
njs: maybe in some OSes ;->
<njs>
dash: for example, the OSes that people use :-P
rokujyouhitoma has quit [Ping timeout: 240 seconds]
<dash>
it's well established that *people* don't care about security, so
<nbjoerg>
njs: just because you know one pointer doesn't mean you know the other
<njs>
I mean, I'm sure it's possible to have some situation where a hack gives access to one arbitrary page of memory but not two arbitrary pages of memory, but I thought the main value in W^X was that it's common to have arbitrary-memory-write bugs like buffer overflows, and rarer to have bugs that let you call mprotect-or-whatever to actually flip protection bits, so forcing exploits to do the latter instead of the former is a win
<nbjoerg>
i.e. consider you can read the current stack frame. that gives you the return address
<nbjoerg>
with W^X you still can't *write* to that location, even if it is a JITed functino
<simpson>
njs: You can weaken an unforgeability requirement to *unguessability*. I recognize that it's cold comfort in memory-unsafe languages.
<njs>
nbjoerg: so this is an ASLR-like argument, pointer leakage of the X pointer doesn't derandomize the W pointer?
<njs>
I'm not actually an exploit writer so I don't really have any useful intuitions about how much of a roadblock these different things are in practice
<njs>
(does the kernel actually use ASLR for arbitrary mmap pages?)
<nbjoerg>
it's better than nothing
<nbjoerg>
and again, a pragmatic solution that depends on X pointer not trivally leaking the W pointer is better than not being able to do W^X at all
<idnar>
njs: if you leak access to both pages you still have a problem; but it's harder to leak both if they're not literally the same page in your address space
vkirilichev has quit [Remote host closed the connection]
<njs>
there are a lot of alleged security hardening features that are "better than nothing" in the sense that they force exploit writers to spend an extra 10 seconds working around them. I'm not saying this is one of them, just that the argument you're making is difficult for me to evaluate :-)
<idnar>
njs: to get arbitrary code execution in the first place, you need some way to a) get your arbitrary code into the process, and then b) execute it; but usually processes do not _intentionally_ execute code that comes from some random external input, so you need to confuse the process into executing some data it previously wrote (eg. stack overflow that jumps into a buffer that has data from the attacker), and it's harder to
<idnar>
accidentally write such code if you don't conflate write and execute
jacob22_ has quit [Ping timeout: 260 seconds]
jcea has quit [Quit: jcea]
<nbjoerg>
sure, strict W^X is certainly better. but it isn't feasible in a lot of situations
<nbjoerg>
so I'm settling with making it possible to use the weaker form where necessary
<nbjoerg>
because otherwise people will go with hacks like the emutramp for selinux
<nbjoerg>
or filesytem wide exceptions like in openbsd
<nbjoerg>
neither is helpful
<idnar>
so if you have something like a JIT compiler, that may need to execute some code that it writes out, but if the JIT compiler is the only thing handling those writeable code pages, then it's much less likely that some other random part of your code will be exploitable in a way that lets you write to them
<idnar>
most attacks do not start off with the ability to write to an arbitrary pointer, you have to bootstrap from some more limited access
<njs>
idnar: I guess the example I'm thinking of is a classic buffer overflow, where you dump some giant NOP slide into memory and then try to guess an address to land in there. In this case you only have to guess the X address anyway, it makes no difference whether it's the same as W.
<njs>
that's probably not at all representative of modern exploits though
<idnar>
njs: it's a lot harder to overflow into some buffer that has nothing to do with the code you're exploiting, though
<idnar>
njs: I mean, if you find an exploit in the JIT compiler itself, then of course it's game over; but to get from, say, a buffer overflow in your HTTP parser, to the JIT compiler's writeable pages, becomes much harder
<idnar>
(not necessarily impossible, of course)
<tbodt>
what if you find an arbitrary read/write primitive in the http parser?
<njs>
idnar: what kind of HTTP parser buffer overflow are you imagining will get you to the JIT compiler's X pages, such that it makes a difference whether or not they're W+X?
<idnar>
njs: if all of your pages are W+X, you don't have to get to the JIT compiler's pages, you can put your executable code in any page
<njs>
idnar: oh sure, I'm assuming that everyone agrees that you want W^X for all your static code and data, and the only question is whether the JIT compiler pages are W+X or are split into W and X views
<Alex_Gaynor>
njs: I haven't thought hard about pypy, but for firefox, W+X pages absolutely make an exploit writers job harder.
<idnar>
tbodt: probably game over then too
<Alex_Gaynor>
It means they don't need to ROP at all, just write shellcode into memory.
<nbjoerg>
W^X you mean?
<idnar>
tbodt: I think (he says, without any actual statistics to hand) that sort of exploit as an initial entrypoint is far less common than more limited vulnerabilities though
<njs>
easier, I think he means
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<njs>
Alex_Gaynor: does firefox simultaneously map the same page as W at one address and X at another address?
<nbjoerg>
I don't know what the limited W^X support in firefox does
<njs>
Alex_Gaynor: my question was specifically whether that's meaningfully different than mapping that same page as W+X at a single address, not about whether W^X is useful in general
<Alex_Gaynor>
njs: I don't know off hand, I'm pretty sure we try to really be W^X though.
<nbjoerg>
but given the nature of limited OS support, they likely just suck up the performance penalty of the mprotect
<Alex_Gaynor>
njs: In a scriptable environment (like a browser!) I'm pretty sure same page mapped at two different locations isn't much a win.
<nbjoerg>
Alex_Gaynor: for 64bit archs, it is fine
tbodt has joined #pypy
<nbjoerg>
Alex_Gaynor: if you don't just randomly ignore segfaults
antocuni has joined #pypy
<Alex_Gaynor>
nbjoerg: is it? At the point where we're doing exploit mitigations, we generally assume the attacker has arbitrary read/write
<nbjoerg>
at that point you don't have to bother since you can pretty much do anything already
<nbjoerg>
(as in: in most program I don't even need to write to executable memory if I can control the rest of (writable) memory)
<njs>
yeah, AFAICT firefox calls mprotect to toggle EXEC_WRITE and EXEC_PROT on and off as needed
<nbjoerg>
.oO(give mozilla a year and some fancy javascript benchmark and they will stop again)
<njs>
(except when running under valgrind because apparently this makes valgrind's jit cranky :-))
asmeurer__ has joined #pypy
<njs>
it looks like v8 toggles between W+X and X, again at a quick look
antocuni has quit [Ping timeout: 248 seconds]
rokujyouhitoma has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
rokujyouhitoma has quit [Ping timeout: 248 seconds]
<Alex_Gaynor>
nbjoerg: With CFI/CFG and some other things we can make it hard for attaker even with arbitrary read/write