[patch] bug in gcc_python_closure_new_generic
by Tom Tromey
Consider this plugin:
class Whatever:
def doit(self, optpass, fun, *args, **kwargs):
print "hi bob"
def main(**kwargs):
w = Whatever()
gcc.register_callback(gcc.PLUGIN_PASS_EXECUTION, w.doit)
When I try to run this (invoking main...), I get an ICE:
*** WARNING *** there are active plugins, do not report this as a bug unless you can reproduce it without enabling any plugins.
Event | Plugins
PLUGIN_FINISH | python
PLUGIN_PASS_EXECUTION | python
../../archer/gdb/python/py-type.c: In function ‘gdbpy_initialize_types’:
../../archer/gdb/python/py-type.c:1378:1: internal compiler error: Segmentation fault
The bug is a missing incref in gcc-python-closure.c. Patch appended.
Tom
diff --git a/gcc-python-closure.c b/gcc-python-closure.c
index 5c9b8ae..f238914 100644
--- a/gcc-python-closure.c
+++ b/gcc-python-closure.c
@@ -39,6 +39,7 @@ gcc_python_closure_new_generic(PyObject *callback, PyObject *extraargs, PyObject
}
closure->callback = callback;
+ Py_INCREF(callback);
// FIXME: we may want to pass in the event enum as well as the user-supplied extraargs
12 years, 5 months
Tracking the range of possible values for an r-value in cpychecker
by David Malcolm
I've slightly reworked how cpychecker tracks unknown values in
http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=commitdiff;h=2...
For integer types, it now uses a new WithinRange abstract value,
describing the closed interval of possible values i.e.:
minvalue <= x <= maxvalue
For example, given this test case:
extern void __cpychecker_dump(int);
char array[12] = {2, 2, 2, 2, 8, 1, 8, 1, 8, 2, 8, 2};
char
test(int i)
{
__cpychecker_dump(i);
/* This ought to constrain i to within [0-255]: */
i = i & 0xff;
__cpychecker_dump(i);
/* and this ought to constrain it to within [0-7]: */
i = i >> 5;
__cpychecker_dump(i);
/* and hence this read is within the array bounds: */
return array[i];
}
we can see the values the analyzer knows about, using the "magic"
function __cpychecker_dump():
__dump((int)val [-0x80000000 <= val <= 0x7fffffff])
__dump((int)val [0 <= val <= 255])
__dump((int)val [0 <= val <= 7])
i.e. it starts by treating "i" as any possible int, but the mask against
0xff gives a new WithinRange(0, 255), and the right-shift constrains it
again.
In theory we can use this to do bounds-checking of array/buffer
accesses. I'm still working through the errors we were discussing in
the "python exceptions from refcount checker" thread, so I'm holding off
on implementing that yet.
I ran into this when compiling a fragment of code similar to the above
in gdb; the difference there is that the array was a local, initialized
within the function scope. cpychecker was complaining that the array
access was a read of an uninitialized value, since it wasn't able to
prove that "i" was within the bounds. Unfortunately, even with this
fix, it still complains, since it isn't smart enough to know that all
possible indices have an initialized value, and picks the "default"
value for the array, which is "uninitialized". Hopefully I can fix that
case too.
Dave
12 years, 5 months
Custom attributes
by David Malcolm
I've just committed initial support for creating custom compilation
attributes from Python, as:
http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=commitdiff;h=a...
So it's now possible to write declarations like these:
void some_function(void)
__attribute__(claims_mutex("io"));
void some_other_function(void)
__attribute__(releases_mutex("io"));
and register those new attributes from Python code, and have a code
analysis script use them to ensure some property (e.g. that the resource
claim/release usage is properly balanced).
See:
http://readthedocs.org/docs/gcc-python-plugin/en/latest/attributes.html
Next step will be to implement some attributes in the cpychecker for
marking up:
* functions that steal references to their PyObject* arguments
* functions that return borrowed references
* functions for which a negative return code indicates that an
exception has been set
Hopefully those will cover the bulk of the false-positives that we're
still seeing on compiling gdb's python code with cpychecker.
Dave
12 years, 5 months
Emulating gcc-xml?
by Alek Paunov
Hi,
Regarding various possible use-cases, Is it currently possible to
emulate/enhance gcc-xml export using a plugin script?
I'd tried, if the data is accessible.
Thank you,
Alek
P.S. Perhaps, it is worth to replicate this plugin too (I love sqlite :-)):
http://code.google.com/p/gccsymdb/
12 years, 5 months
Obtaining absolute paths when obtaining locations
by satya komaragiri
Hi,
I am trying to use gcc-python-plugin to obtain caller-callee
information in JSON format and I need to obtain the location as well.
I am using callgraphnode.decl.location and edge.caller.decl.location
for this. When the file is located in the current directory , I get
relative paths.
Is there any way to always obtain absolute paths?
Regards,
Satya
12 years, 5 months
gcc-with-cpychecker now performs refcount checking
by David Malcolm
As of commit 7cd150fe609b606f259af18fc98d34b4fbde20a7, the
gcc-with-cpychecker harness now invokes the reference-count checker,
removing the need to hack up libcpychecker/__init__.py to do this.
So now you can invoke the checker on arbitrary CPython extension code
(e.g. gdb) with something like:
make CC=/path/to/built/plugin/gcc-with-cpychecker
without having to hack up the checker's code: you'll get a GCC with the
extra "cpychecker" compilation pass, which runs these tests:
* format checking of Py_ArgParse* and Py_BuildValue calls
* reference count analysis
Having said that, expect to run into tracebacks and other bugs with the
checker if you do (including incorrect reports - so proceed with
caution!). I'm still slowly working through gdb and the plugin's code
with the checker, fixing issues as I see them.
Adding some kind of custom attributes to mark functions that steal
references (and those that return borrowed references) is probably next.
Dave
12 years, 5 months
Success stories
by David Malcolm
I've gone ahead and created a "Success Stories" page within the plugin's
documentation (docs/success.rst).
I took the liberty of adding links to the various bugs we've been
finding in gdb - I hope that's OK - I don't want to make it appear that
I'm picking on gdb (sorry!), but I did want to capture the benefit of
the plugin in one place. We can reword things, of course.
You can see the HTML form of the page here:
http://readthedocs.org/docs/gcc-python-plugin/en/latest/success.html
[I hope that tracking all of this manually will eventually become
unmanageable, which would be, as they say, a good problem to have]
I'll expand that page to add the various bugs that the plugin shows up
in itself, though I'm not sure that really counts as a "success" :)
Has anyone else found the plugin useful so far?
Dave
(goes back to fixing bugs in cpychecker that compiling gdb shows up)
12 years, 5 months
python exceptions from refcount checker
by Tom Tromey
I tried out the refcount checker on gdb today. I'm using git head as of
a few minutes ago, with a newly-updated F15 system gcc.
I hacked cpychecker.py:
diff --git a/cpychecker.py b/cpychecker.py
index e120c53..4c24e3f 100644
--- a/cpychecker.py
+++ b/cpychecker.py
@@ -16,4 +16,4 @@
# <http://www.gnu.org/licenses/>.
from libcpychecker import main
-main()
+main(verify_refcounting = True)
(Note that this patch causes test suite failures -- but I think that
can't be very important.)
I wrote this little script to run 'make' with the right options:
#!/bin/sh
here=/home/tromey/Space/Trunk/gcc-python-plugin/
make CFLAGS="-gdwarf-4 -g3 -fplugin=$here/python.so -fplugin-arg-python-script=$here/cpychecker.py" "$@"
Then I ran it on gdb like:
/tmp/Makeit -k -C ~/gnu/archer/build/gdb -j2
I get many failures like this:
../../archer/gdb/armbsd-tdep.c: In function ‘armbsd_regset_from_core_section’:
../../archer/gdb/armbsd-tdep.c:127:3: error: Unhandled Python exception raised calling 'execute' method
Traceback (most recent call last):
File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/__init__.py", line 49, in execute
self.show_possible_null_derefs)
File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/refcounts.py", line 2020, in check_refcounts
limits=Limits(maxtrans=1024))
File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/absinterp.py", line 1821, in iter_traces
f_new = facet_cls(curstate, fun=fun)
File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/refcounts.py", line 224, in __init__
self.exception_rvalue = ConcreteValue(get_PyObjectPtr(),
File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/types.py", line 60, in get_PyObjectPtr
return get_global_typedef('PyObject').pointer
AttributeError: 'NoneType' object has no attribute 'pointer'
../../archer/gdb/armbsd-tdep.c: In function ‘armbsd_regset_from_core_section’:
../../archer/gdb/armbsd-tdep.c:127:3: error: Unhandled Python exception raised calling 'execute' method
Traceback (most recent call last):
File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/__init__.py", line 49, in execute
self.show_possible_null_derefs)
File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/refcounts.py", line 2020, in check_refcounts
limits=Limits(maxtrans=1024))
File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/absinterp.py", line 1821, in iter_traces
f_new = facet_cls(curstate, fun=fun)
File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/refcounts.py", line 224, in __init__
self.exception_rvalue = ConcreteValue(get_PyObjectPtr(),
File "/home/tromey/Space/Trunk/gcc-python-plugin/libcpychecker/types.py", line 60, in get_PyObjectPtr
return get_global_typedef('PyObject').pointer
AttributeError: 'NoneType' object has no attribute 'pointer'
The problem here is that some files, like this one, do not use Python at
all, and don't include the Python headers. In this situation the plugin
dies when it fails to look up a Python type.
Tom
12 years, 5 months
FYI: const fix in py-type.c
by Tom Tromey
I'm checking this in.
David Malcolm's GCC plugin found a bug in py-type.c -- we were using
PyArg_ParseTuple's "s" format but passing in a char*, not a const char*.
The fix is appended.
Thanks again to David for this plugin.
Tom
2011-10-13 Tom Tromey <tromey(a)redhat.com>
* python/py-type.c (typy_has_key): Make 'field' const.
Index: python/py-type.c
===================================================================
RCS file: /cvs/src/src/gdb/python/py-type.c,v
retrieving revision 1.24
diff -u -r1.24 py-type.c
--- python/py-type.c 9 Oct 2011 19:21:38 -0000 1.24
+++ python/py-type.c 13 Oct 2011 13:48:41 -0000
@@ -1176,7 +1176,7 @@
typy_has_key (PyObject *self, PyObject *args)
{
struct type *type = ((type_object *) self)->type;
- char *field;
+ const char *field;
int i;
if (!PyArg_ParseTuple (args, "s", &field))
12 years, 5 months