Newbie questions
by Stephan Bergmann
Hi all,
Playing around with the plugin (in the hope of eventually using it to
find out interesting things in the LibreOffice code base), I'm not sure
I'm trying to use it in a sensible manner. Maybe somebody on this list
can set me on track?
As a first experiment, I wanted to do the following: LO has a string
class rtl::OUString with member functions getLength and isEmpty. Now, I
wanted to find places in the code that call getLength and check the
result against null, instead of calling isEmpty directly.
My first doubt is whether doing that at Gimple-level is practical at all
(and whether the plugin would also support doing it at another, more
appropriate level that is closer to the original AST)?
I figure I need to find GimpleCall statements that call
rtl::OUString::getLength and trace their lhs variables (potentially
through further GimpleAssign statements) to GimpleCond statements.
Something like
class TryOut(gcc.GimplePass):
def execute(self, fun):
for bb in fun.cfg.basic_blocks:
if bb.gimple:
for stmt in bb.gimple:
print ' stmt:', stmt
if isinstance(stmt, gcc.GimpleCall):
name = stmt.fndecl.name
# ...
elif isinstance(stmt, gcc.GimpleAssign):
# ...
elif isinstance(stmt, gcc.GimpleCond):
# ...
But the first obstacle is already how to determine the full name of the
called function, as stmt.fndecl.name would only be "getLength", without
the class and namespace information.
Any pointers appreciated,
Stephan
12 years, 4 months
ANN: gcc-python-plugin 0.7
by David Malcolm
gcc-python-plugin is a plugin for GCC 4.6 onwards which embeds the
CPython interpreter within GCC, allowing you to write new compiler
warnings in Python, generate code visualizations, etc.
It ships with "cpychecker", which implements static analysis passes for
GCC aimed at finding bugs in CPython extensions. In particular, it can
automatically detect reference-counting errors:
http://gcc-python-plugin.readthedocs.org/en/latest/cpychecker.html
Highlights of this release:
* You can now create custom GCC attributes from Python, allowing you
to add custom high-level annotations to a C API, and to write Python
scripts that will verify these properties.
* The "cpychecker" code has too many improvements to list here, and
has now detected many reference-counting bugs in real-world CPython
extension code. Some success stories can be seen at:
http://gcc-python-plugin.readthedocs.org/en/latest/success.html
The usability and signal:noise ratio is greatly improved over
previous releases. It can now emit HTML reports showing the path of
execution through a function that triggers a particular error.
For more information you can read detailed notes on this release at:
http://readthedocs.org/docs/gcc-python-plugin/en/latest/0.7.html
Tarball releases are available at:
https://fedorahosted.org/releases/g/c/gcc-python-plugin/
Prebuilt-documentation can be seen at:
http://gcc-python-plugin.readthedocs.org/en/latest/index.html
The project's homepage is:
https://fedorahosted.org/gcc-python-plugin/
The plugin and checker are Free Software, licensed under the GPLv3 or
later. Thanks to Red Hat for funding their development.
If you're using Fedora, pre-built RPMs of the plugin should be hitting
the repositories shortly.
Enjoy!
Dave Malcolm
12 years, 4 months
Deduplication of error reports in cpychecker
by David Malcolm
cpychecker tries to track every possible trace of execution through a
function, and so it can sometimes spam you with large numbers of
seemingly duplicate error reports for a single underlying root cause.
So I've implemented some simple deduplication of error reports within
the cpychecker (as of a90611cfbd57a75c80351f3250e2388255e3943b in git).
The new code gathers up error reports that are sufficiently similar to
each other ("equivalence classes" if you want some mathematical jargon).
For now the criteria are a little simplistic: two reports are equivalent
if they are for the same function, at the same location, and with
the same summary text.
It only issues a detailed report for the first error it sees in each
partition, together with a note on how many other similar errors it saw.
This seems to improve the signal:noise ratio of cpychecker: it
successfully removes a genuine duplicate within the test suite [1]
and removes quite a few genuine duplicates when recompiling the plugin
using gcc-with-cpychecker, making that output much easier to read.
But if it seems to be collapsing reports together too aggressively we
could tune it a bit, I guess (e.g. by adding some hint fields to error
reports for the deduplication code to compare).
Dave
[1] tests/cpychecker/refcounts/PyDict_SetItem/incorrect
12 years, 4 months
Failing "make" self-tests
by Stephan Bergmann
Hi all,
With a recent gcc-python-plugin.git repo (as of last Friday), doing
"make" failed a lot of the included self-tests (see the attached
log.txt). The resulting plugin appears to work just fine, however.
This was done on a Fedora-16 x86_64 machine with
gcc-4.6.2-1.fc16.x86_64
gcc-plugin-devel-4.6.2-1.fc16.x86_64
python-2.7.2-5.2.fc16.x86_64
python-devel-2.7.2-5.2.fc16.x86_64
python3-3.2.1-1.fc16.x86_64
(Maybe the existence of python3 is a problem? I'm no big Python user,
don't know if those two interfere with each other in any way?)
Stephan
12 years, 4 months
Associating run-time and compile-time type information
by David Malcolm
I added a new custom compiler attribute to the cpychecker, which allows
you to associate a PyTypeObject instance with a C typedef:
This means that the cpychecker can now tie together run-time type
information with C's compile-time type information:
/* Define some PyObject subclass, as both a struct and a typedef */
struct OurObjectStruct {
PyObject_HEAD
/* other fields */
};
typedef struct OurObjectStruct OurExtensionObject;
/*
Declare the PyTypeObject (run-time type info), using the custom
attribute to associate it with the typedef above:
*/
extern PyTypeObject UserDefinedExtension_Type
CPYCHECKER_TYPE_OBJECT_FOR_TYPEDEF("OurExtensionObject");
The checker uses this so far in just one specific place, but it could be
used for various other verifications (e.g. to check that casts are
sane).
This is in
http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=commit;h=ff3eb...
See:
http://gcc-python-plugin.readthedocs.org/en/latest/cpychecker.html#associ...
Naturally the plugin itself now uses this attribute, for the case where
the checker is verifying the plugin; doing so fixes some false
positives. This is in git in 223bccfc32603be851e0d9b62aee8fd999c329eb.
Dave
12 years, 5 months
gcc plugin for checking gdb's exception handlers
by Tom Tromey
I wrote a GCC plugin, using David Malcolm's excellent Python plugin, to
check for proper use of gdb exceptions in the Python code. I added new
'gdb_throw' and 'gdb_nothrow' attributes and marked a couple spots in
the sources. The plugin examines all the calls in a file and writes out
a sort of call graph as Python code. Then, after rebuilding GDB with
the plugin in place, there is a second program that reads all the data,
propagates the "can-throw" property through the call graph, and reports
errors whenever a nothrow function is discovered to throw. It reports
the path from the function to the throw, like this:
../../archer/gdb/python/python.c:481: error: function gdbpy_decode_line is marked nothrow but can throw
../../archer/gdb/python/python.c:497: info: via call to get_current_arch
../../archer/gdb/arch-utils.c:758: info: via call to get_selected_frame
../../archer/gdb/frame.c:1360: info: via call to error
../../archer/gdb/utils.c:830: info: via call to throw_verror
I also had it mark cleanup functions as 'nothrow', since I thought that
is either the rule or what the rule should be. This showed a number of
errors, enough that I am not sure what I want to do about it yet.
I'm attaching the plugin and the analysis program. If you run it
yourself, please note that it still emits some false reports. You must
examine each one, especially the ones arising from indirect function
calls, to see whether it is real.
I plan to run it periodically to check for new bugs. To that end I will
probably put in the needed gdb patch, also attached.
I filed a bug for all the reports in the Python code.
Tom
12 years, 5 months
[patch] bug in gcc_python_closure_new_generic
by Tom Tromey
Consider this plugin:
class Whatever:
def doit(self, optpass, fun, *args, **kwargs):
print "hi bob"
def main(**kwargs):
w = Whatever()
gcc.register_callback(gcc.PLUGIN_PASS_EXECUTION, w.doit)
When I try to run this (invoking main...), I get an ICE:
*** WARNING *** there are active plugins, do not report this as a bug unless you can reproduce it without enabling any plugins.
Event | Plugins
PLUGIN_FINISH | python
PLUGIN_PASS_EXECUTION | python
../../archer/gdb/python/py-type.c: In function ‘gdbpy_initialize_types’:
../../archer/gdb/python/py-type.c:1378:1: internal compiler error: Segmentation fault
The bug is a missing incref in gcc-python-closure.c. Patch appended.
Tom
diff --git a/gcc-python-closure.c b/gcc-python-closure.c
index 5c9b8ae..f238914 100644
--- a/gcc-python-closure.c
+++ b/gcc-python-closure.c
@@ -39,6 +39,7 @@ gcc_python_closure_new_generic(PyObject *callback, PyObject *extraargs, PyObject
}
closure->callback = callback;
+ Py_INCREF(callback);
// FIXME: we may want to pass in the event enum as well as the user-supplied extraargs
12 years, 5 months