Functions setting exceptions
by Daniele Varrazzo
Hello,
attached there is a patch to implement PySequence_SetItem, which
*doesn't* steal a reference as the PyList/PyTuple variants do, and
returns 0 on success, -1 on error.
After cleaning many genuine errors in psycopg, mostly about unchecked
return values, I'm left with several false positives related exactly
to this type of functions, returning 0 on success, -1 on failure and
setting an exception. Each of these functions caller is reported as
"returning (PyObject*)NULL without setting an exception".
Is there any way to flag the behaviour of these functions? An
__attribute__? I'm trying implementing it but time is out for tonight.
I've find how to register the attribute callback but haven't found yet
where and how to use it.
-- Daniele
12 years
analyzing psycopg2
by Daniele Varrazzo
Hello,
I'm using the CPython static analyzer on psycopg2 and it's being quite
helpful: so far it has discovered several quirks. They are mostly
relative to unlikely code paths, such as out of memory during module
import, nonetheless I'm trying to fix everything. Here is something to
report:
1. Running the repos head out-of-the-box, the program stops quite
early with an error. The attached steal-ref.patch resolves it, but I
don't know if the fix is correct.
2. The library uses PyWeakref_GetObject, returning a borrowed ref:
I've implemented it, with tests, in the other attached patch.
3. The run still doesn't complete: after a lot of output it tracebacks with:
....
File "/home/piro/fs/gcc-python-plugin/libcpychecker/absinterp.py",
line 2451, in eval_rhs
return self.eval_rvalue(rhs[0], stmt.loc)
File "/home/piro/fs/gcc-python-plugin/libcpychecker/absinterp.py",
line 1498, in eval_rvalue
region = self.get_field_region(expr, loc)#.target, expr.field.name)
File "/home/piro/fs/gcc-python-plugin/libcpychecker/absinterp.py",
line 1660, in get_field_region
self.raise_split_value(ptr)
File "/home/piro/fs/gcc-python-plugin/libcpychecker/absinterp.py",
line 1984, in raise_split_value
check_isinstance(ptr_rvalue.gcctype, gcc.PointerType)
File "/home/piro/fs/gcc-python-plugin/gccutils.py", line 629, in
check_isinstance
raise TypeError('%s / %r is not an instance of %s' % (obj, obj, types))
TypeError: None / None is not an instance of <type 'gcc.PointerType'>
I haven't debugged it yet: will try when I arrive to that point, for
the moment I'm fixing psycopg instead.
4. Another weird function psycopg uses is _PyString_Resize
(_PyBytes_Resize in Python 3): the function looks no more public (it
was in the API docs for Python 2.0), and is used in the
reimplementation of the % operator for bytes that psycopg uses
internally (see [1], as _Bytes_Resize). This weird function takes a
PyString as input and returns 0 on success, else returns -1 *and
steals a reference*. The quirkness is reported in [2]. I guess it
would take a "CPython.make_transitions_for_success_or_steal()" method,
but haven't seen yet how to implement it.
5. While at it, make_transitions_for_borrowed_ref_or_fail docstring
says it takes an optional name: it's probably a pasto from the above
docstring.
6. Finally, haven't been able to create a ticket to
https://fedorahosted.org/gcc-python-plugin/: I should probably log in
but can't find how to create an account, unless opening a project on
fedora. Maybe I'm just sleepy.
Will try to provide further patches as I go ahead cleaning up psycopg.
The branch where I'm cleaning up is at [3] (in case you want to
reproduce the reported issues).
Thank you very much for the tool: it seems very useful.
-- Daniele
[1] https://github.com/dvarrazzo/psycopg/blob/devel/psycopg/bytes_format.c
[2] http://mail.python.org/pipermail/python-dev/2002-April/023719.html
[3] https://github.com/dvarrazzo/psycopg/commits/gcc-python-plugin
12 years, 1 month
Triaging script for gcc-with-cpychecker
by David Malcolm
I've been working on my primary use-case for the gcc-python-plugin:
finding bugs in C Python extension code.
To that end, I've been attempting to automate a mass rebuild of all of
the C Python extension code in Fedora, building it with the cpychecker
code [1]. The source tree has recently grown a "misc/fedora"
subdirectory, and within it there's a mass-rebuild.py script which
attempts to rebuild a set of source packages (within chroots), injecting
the gcc python plugin and the libcpychecker code into the build. It
then slurps out all of the refcount-error.html files from the chroot
into a results directory.
On doing so, I'm running into the very real issue of what to do with all
the errors that the plugin finds: some packages have dozens of errors.
I want to provide something that an upstream maintainer of a python
package can look at and understand, without subjecting them to a wall of
noise.
To that end, I've built a triaging script which walks over the results,
and tries to triage them into high-severity through low-severity
categories, and writes out an index.html indexing them all.
You can see an example of all of this here, where the cpychecker code
has analyzed the python bindings for GStreamer:
http://people.fedoraproject.org/~dmalcolm/gcc-python-plugin/2012-02-10/gs...
Thoughts?
Hope this looks sane
Dave
[1] I'm tracking this for Fedora here:
http://fedoraproject.org/wiki/Features/StaticAnalysisOfPythonRefcounts
12 years, 1 month
Dominator Analysis
by Emílio Wuerges
Hello,
I've made a small patch including my dominator analysis.
It is ugly.
It is the first time I did something with python embedded in C, so I'm
aware I did not use the best practices.
I inserted all my changes inside the gcc-python.c
If you could check my patch and suggest some changes I'd be happy to
do them before re-submitting my patch.
--
Emilio Wuerges
ECL - Embedded Computing Lab
UFSC - Universidade Federal de Santa Catarina
Brasil
12 years, 1 month
Small patch for x86_64 compatibility
by Emílio Wuerges
Hello, I starting to use the python plugin today.
In order to build it, I had to 2 two things:
1) add -fPIC to testcpybuilder.py compile flags. (in the patch)
2) to use the plugin, I have to add a LD_PRELOAD=libpython2.7.so
I don't know if that was intended, but maybe it would be nice to add a
hint to the README.
I'm using ubuntu oneiric ocelot 64bit.
Also I got some failures in some tests:
253 successes; 4 failures; 1 skipped
Failed tests:
tests/plugin/gc/_gc_selftest
tests/plugin/gimple-walk-tree/dump-all
tests/plugin/gimple-walk-tree/exceptions
tests/plugin/gimple-walk-tree/find-one
make: *** [test-suite] Error 1
3 of them had the same reason, just some warnings:
--- Expected stderr
+++ Actual stderr
@@ -0,0 +1,3 @@
+tests/plugin/gc/_gc_selftest/input.c: In function 'main':
+tests/plugin/gc/_gc_selftest/input.c:41:9: warning: format '%i'
expects argument of type 'int', but argument 2 has type 'char *'
[-Wformat]
+tests/plugin/gc/_gc_selftest/input.c:41:9: warning: format '%s'
expects a matching 'char *' argument [-Wformat]
1 of them had this reason, with an actual error:
--- Expected stderr
+++ Actual stderr
@@ -1,4 +1,6 @@
tests/plugin/gimple-walk-tree/exceptions/input.c: In function 'main':
+tests/plugin/gimple-walk-tree/exceptions/input.c:41:9: warning:
format '%i' expects argument of type 'int', but argument 2 has type
'char *' [-Wformat]
+tests/plugin/gimple-walk-tree/exceptions/input.c:41:9: warning:
format '%s' expects a matching 'char *' argument [-Wformat]
tests/plugin/gimple-walk-tree/exceptions/input.c:35:1: error:
Unhandled Python exception raised calling 'execute' method
Traceback (most recent call last):
File "tests/plugin/gimple-walk-tree/exceptions/script.py", line 30,
in execute
tests/plugin/gimple-walk-tree/find-one: FAIL
--
Emilio Wuerges
ECL - Embedded Computing Lab
UFSC - Universidade Federal de Santa Catarina
Brasil
12 years, 1 month
Initial support for GCC 4.7
by David Malcolm
I've committed some fixes that make the plugin work with GCC 4.7 (or, at
least, the gcc-4.7.0-0.2.fc17.x86_64 on my test box).
Doing this [1] requires a configuration mechanism, so that we can detect
the features of the gcc we're building against at the beginning of the
build, and conditionalize things accordingly.
So the Makefile has gained a new step in which an autogenerated-config.h
is created; the output deliberately mimics that of an autoconf-generated
"configure" script, but the configurator is written in Python, rather
than m4.
Hopefully this doesn't break the build for anyone.
Dave
[1] gory details: gcc.Block.vars no longer contains unused globals in
gcc 4.7 (see https://fedorahosted.org/gcc-python-plugin/ticket/21 ),
which broke the cpychecker code's usage of various CPython globals
during its analyses (type objects such as "PyList_Type" and exception
objects such as "PyExc_MemoryError"). It now works around this by using
the new gcc.PLUGIN_FINISH_DECL event to get at these as they're parsed,
if that event is available, otherwise doing it the old way.
12 years, 1 month
fix issue #31
by Tom Tromey
This fixes issue #31 by documenting a couple of fields of IntegerType.
The bug is just that min_value and max_value are not documented.
I looked and all the other fields are documented.
It would be nice if there were a way to verify that each field or method
is also documented, so that "make" would fail if something were missing.
Offhand I am not sure how best to do this.
Tom
>From 51de57a5b9ddb150d5eceee9332538caca87c7b1 Mon Sep 17 00:00:00 2001
From: Tom Tromey <tromey(a)redhat.com>
Date: Tue, 24 Jan 2012 13:49:36 -0700
Subject: [PATCH 2/2] Fix issue #31
---
docs/tree.rst | 8 ++++++++
1 files changed, 8 insertions(+), 0 deletions(-)
diff --git a/docs/tree.rst b/docs/tree.rst
index cd3090f..d3798ad 100644
--- a/docs/tree.rst
+++ b/docs/tree.rst
@@ -425,6 +425,14 @@ Types
The gcc.IntegerType for the unsigned version of this type
+ .. py:attribute:: max_value
+
+ The maximum possible value for this type, as a `gcc.IntegerCst`
+
+ .. py:attribute:: min_value
+
+ The minimum possible value for this type, as a `gcc.IntegerCst`
+
.. py:class:: gcc.FloatType
Subclass of :py:class:`gcc.Type` representing C's `float` and `double` types
--
1.7.6.5
12 years, 1 month
fix issue #6
by Tom Tromey
This "fixes" issue #6 by documenting the current state.
The bug requests a way to call define_macro with two arguments, but
David pointed out that define_macro("NAME=VALUE") already works.
Tom
>From 4977934000d92c107d8d779476a2bfa2d84f9977 Mon Sep 17 00:00:00 2001
From: Tom Tromey <tromey(a)redhat.com>
Date: Tue, 24 Jan 2012 13:29:16 -0700
Subject: [PATCH 1/2] Fix issue #6 by documenting that define_macro accepts a
definition as well.
---
docs/preprocessor.rst | 13 ++++++++++---
1 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/docs/preprocessor.rst b/docs/preprocessor.rst
index 7e37f03..e4110f3 100644
--- a/docs/preprocessor.rst
+++ b/docs/preprocessor.rst
@@ -7,10 +7,17 @@ For languages that support a preprocessor, it's possible to inject new
The motivation for this is to better support the creation of custom
attributes, by creating preprocessor names that can be tested against.
-.. py:function:: gcc.define_macro(name)
+.. py:function:: gcc.define_macro(argument)
- Defines a preprocessor macro with the given name, which may be of use
- for code that needs to test for the presence of your script.
+ Defines a preprocessor macro with the given argument, which may be
+ of use for code that needs to test for the presence of your script.
+ The argument can either be a simple name, or a name with a
+ definition:
+
+ .. code-block:: python
+
+ gcc.define_macro("SOMETHING") # define as the empty string
+ gcc.define_macro("SOMETHING=72")
This function can only be called from within specific event callbacks,
since it manipulates the state of the preprocessor for a given source
--
1.7.6.5
12 years, 1 month