7:43 p.m.
I've just committed initial support for creating custom compilation
attributes from Python, as:
http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=commitdiff;h=a...
So it's now possible to write declarations like these:
void some_function(void)
__attribute__(claims_mutex("io"));
void some_other_function(void)
__attribute__(releases_mutex("io"));
and register those new attributes from Python code, and have a code
analysis script use them to ensure some property (e.g. that the resource
claim/release usage is properly balanced).
See:
http://readthedocs.org/docs/gcc-python-plugin/en/latest/attributes.html
Next step will be to implement some attributes in the cpychecker for
marking up:
* functions that steal references to their PyObject* arguments
* functions that return borrowed references
* functions for which a negative return code indicates that an
exception has been set
Hopefully those will cover the bulk of the false-positives that we're
still seeing on compiling gdb's python code with cpychecker.
Dave
6:20 a.m.
On Mon, 2011-10-24 at 20:43 -0400, David Malcolm wrote:
I've just committed initial support for creating custom
compilation
attributes from Python, as:
http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=commitdiff;h=a...
Nice!
So it's now possible to write declarations like these:
void some_function(void)
__attribute__(claims_mutex("io"));
void some_other_function(void)
__attribute__(releases_mutex("io"));
and register those new attributes from Python code, and have a code
analysis script use them to ensure some property (e.g. that the resource
claim/release usage is properly balanced).
According to http://gcc.gnu.org/onlinedocs/gcc/Attribute-Syntax.html two
open and two closing brackets are needed for the __attribute__ syntax
e.g.
void some_function(void)
__attribute__((claims_mutex("io")));
(Same issue on the attributes.html doc.)
Is there a way to detect whether or not you are compiling with a
particular python plugin loaded in the source code to shut up the
warnings about none-existing attribute names if you are not by defining
things like:
#ifdef COMPILING_AGAINST_FOOBAR_ATTR_PYTHON_CHECKER
#define FOOBAR_ATTR __attribute__((foobar))
#else
#define FOOBAR_ATTR
#endif
I can kind of guess what the decl_required, type_required,
function_type_required are for, but they aren't documented in the above.
Thanks,
Mark
10:31 a.m.
On Tue, 2011-10-25 at 13:20 +0200, Mark Wielaard wrote:
On Mon, 2011-10-24 at 20:43 -0400, David Malcolm wrote:
[...snip...]
According to http://gcc.gnu.org/onlinedocs/gcc/Attribute-Syntax.html
two
open and two closing brackets are needed for the __attribute__ syntax
e.g.
void some_function(void)
__attribute__((claims_mutex("io")));
(Same issue on the attributes.html doc.)
Thanks! I've fixed this in 4c6ff87c5d32867f3d12534815f35e88cc2a1a32
Is there a way to detect whether or not you are compiling with a
particular python plugin loaded in the source code to shut up the
warnings about none-existing attribute names if you are not by defining
things like:
#ifdef COMPILING_AGAINST_FOOBAR_ATTR_PYTHON_CHECKER
#define FOOBAR_ATTR __attribute__((foobar))
#else
#define FOOBAR_ATTR
#endif
I'm not sure of the best way of doing this. My first thought was "use
some kind of configuration test", but that could be a nuisance,
especially if you want to only compile some subset of the code with the
checker.
Idea: have the plugin predefine a preprocessor macro. I don't yet know
whether or not a gcc plugin can hook into the preprocessor, and provide
some predefined macros. Will investigate...
> See:
> http://readthedocs.org/docs/gcc-python-plugin/en/latest/attributes.html
I can kind of guess what the decl_required, type_required,
function_type_required are for, but they aren't documented in the above.
I directly wrapped GCC's register_attribute() API here, and I don't have
a good sense of what they're for (but I wanted to get the code off of my
laptop and into git, rather than wait until it was perfect :) ). I'm
new to attributes in GCC.
Hope this is helpful
Dave
11:42 a.m.
David> I'm not sure of the best way of doing this. My first thought was "use
David> some kind of configuration test", but that could be a nuisance,
David> especially if you want to only compile some subset of the code with the
David> checker.
Also it would mean reconfiguring just to try out the plugin.
That seems like a pain.
David> I directly wrapped GCC's register_attribute() API here, and I don't
have
David> a good sense of what they're for (but I wanted to get the code off of my
David> laptop and into git, rather than wait until it was perfect :) ). I'm
David> new to attributes in GCC.
The comments in tree.h:struct attribute_spec are pretty good:
/* Whether this attribute requires a DECL. If it does, it will be passed
from types of DECLs, function return types and array element types to
the DECLs, function types and array types respectively; but when
applied to a type in any other circumstances, it will be ignored with
a warning. (If greater control is desired for a given attribute,
this should be false, and the flags argument to the handler may be
used to gain greater control in that case.) */
const bool decl_required;
/* Whether this attribute requires a type. If it does, it will be passed
from a DECL to the type of that DECL. */
const bool type_required;
/* Whether this attribute requires a function (or method) type. If it does,
it will be passed from a function pointer type to the target type,
and from a function return type (which is not itself a function
pointer type) to the function type. */
const bool function_type_required;
Tom
3:14 p.m.
On Tue, 2011-10-25 at 11:31 -0400, David Malcolm wrote:
On Tue, 2011-10-25 at 13:20 +0200, Mark Wielaard wrote:
> On Mon, 2011-10-24 at 20:43 -0400, David Malcolm wrote:
...
> Is there a way to detect whether or not you are compiling with
a
> particular python plugin loaded in the source code to shut up the
> warnings about none-existing attribute names if you are not by defining
> things like:
>
> #ifdef COMPILING_AGAINST_FOOBAR_ATTR_PYTHON_CHECKER
> #define FOOBAR_ATTR __attribute__((foobar))
> #else
> #define FOOBAR_ATTR
> #endif
I'm not sure of the best way of doing this. My first thought was "use
some kind of configuration test", but that could be a nuisance,
especially if you want to only compile some subset of the code with the
checker.
Idea: have the plugin predefine a preprocessor macro. I don't yet know
whether or not a gcc plugin can hook into the preprocessor, and provide
some predefined macros. Will investigate...
I managed to do this: as of:
http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=commit;h=2f917...
there's now a gcc.define_macro() hook, so you can define the macro in
your python script at the same time as you define the attribute.
I added an example of how to do this to the docs here:
http://readthedocs.org/docs/gcc-python-plugin/en/latest/attributes.html#u...
I'll start adding some custom attributes to the cpychecker code next, to
use all of this machinery, and have a go at using them in gdb to silence
some of the false warnings.
6:16 p.m.
On Tue, 2011-10-25 at 16:14 -0400, David Malcolm wrote:
On Tue, 2011-10-25 at 11:31 -0400, David Malcolm wrote:
> On Tue, 2011-10-25 at 13:20 +0200, Mark Wielaard wrote:
> > On Mon, 2011-10-24 at 20:43 -0400, David Malcolm wrote:
...
> > Is there a way to detect whether or not you are compiling with a
> > particular python plugin loaded in the source code to shut up the
> > warnings about none-existing attribute names if you are not by defining
> > things like:
> >
> > #ifdef COMPILING_AGAINST_FOOBAR_ATTR_PYTHON_CHECKER
> > #define FOOBAR_ATTR __attribute__((foobar))
> > #else
> > #define FOOBAR_ATTR
> > #endif
>
> I'm not sure of the best way of doing this. My first thought was "use
> some kind of configuration test", but that could be a nuisance,
> especially if you want to only compile some subset of the code with the
> checker.
>
> Idea: have the plugin predefine a preprocessor macro. I don't yet know
> whether or not a gcc plugin can hook into the preprocessor, and provide
> some predefined macros. Will investigate...
I managed to do this: as of:
http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=commit;h=2f917...
there's now a gcc.define_macro() hook, so you can define the macro in
your python script at the same time as you define the attribute.
I added an example of how to do this to the docs here:
http://readthedocs.org/docs/gcc-python-plugin/en/latest/attributes.html#u...
I'll start adding some custom attributes to the cpychecker code next, to
use all of this machinery, and have a go at using them in gdb to silence
some of the false warnings.
I've added the first of these to the cpychecker code, as of:
http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=commitdiff;h=3...
Tom: I'm attaching a patch for gdb which gives an example of how this
might be used for gdb's Python code.
The patch isn't ready as-is, but hopefully gives an idea of what's
possible, and takes another bite out of the big wall of false positives
that the checker currently emits.
Ideally, the macros and attributes probably ought to folded in to
Python's headers (which will require discussion with upstream Python).
Hope this looks sane
Dave
9:34 a.m.
>>>> "David" == David Malcolm
<dmalcolm(a)redhat.com> writes:
David> Tom: I'm attaching a patch for gdb which gives an example of how this
David> might be used for gdb's Python code.
Awesome.
David> The patch isn't ready as-is, but hopefully gives an idea of what's
David> possible, and takes another bite out of the big wall of false positives
David> that the checker currently emits.
The patch seems pretty close to me.
I looked at this this morning, before I read your mail, and came up with
nearly the same thing :)
If you want, we can make a branch in archer.git for this sort of thing.
Then once you think the plugin is really ready, we can push it all
upstream into gdb. Let me know; I can give you write access or set up
the branch or whatever.
David> + #define CPYCHECKER_RETURNS_BORROWED_REF() \
David> + __attribute__((cpychecker_returns_borrowed_ref))
The gdb style is to use object-like, not function-like, macros for this
kind of thing.
E.g., from src/include/ansidecl.h:
#if (GCC_VERSION < 2007)
# define __attribute__(x)
#endif
[...]
#ifndef ATTRIBUTE_UNUSED
#define ATTRIBUTE_UNUSED __attribute__ ((__unused__))
#endif /* ATTRIBUTE_UNUSED */
David> +#else
David> + #define CPYCHECKER_RETURNS_BORROWED_REF()
David> + #error (This should have been defined)
David> +#endif
I think having a #error here is wrong.
It is ok not to use the plugin ;-)
Tom
1:40 p.m.
On Wed, 2011-10-26 at 08:34 -0600, Tom Tromey wrote:
>>>>> "David" == David Malcolm
<dmalcolm(a)redhat.com> writes:
...
David> The patch isn't ready as-is, but hopefully gives an
idea of what's
David> possible, and takes another bite out of the big wall of false positives
David> that the checker currently emits.
The patch seems pretty close to me.
I looked at this this morning, before I read your mail, and came up with
nearly the same thing :)
:)
If you want, we can make a branch in archer.git for this sort of
thing.
Then once you think the plugin is really ready, we can push it all
upstream into gdb. Let me know; I can give you write access or set up
the branch or whatever.
I was going to say "sure", but I'm a bit nervous about this - I don't
want to get into the business of actually fixing gdb's bugs.
I want to provide the tools to let developers easily identify various
categories of bugs that were hard to spot before - but I don't want to
actually have to do the work of fixing those bugs - if that makes sense.
(I don't want to end up having to fix every reference-counting bug in
every package on PyPI)
So I'd be happy to put these additions into a branch, but I'm hoping my
direct code contributions to gdb for this will be a relatively small
patch to python-internal.h
(Hope this doesn't come across wrong)
David> + #define CPYCHECKER_RETURNS_BORROWED_REF() \
David> + __attribute__((cpychecker_returns_borrowed_ref))
The gdb style is to use object-like, not function-like, macros for this
kind of thing.
By "object-like", do you mean without parentheses?
E.g., from src/include/ansidecl.h:
#if (GCC_VERSION < 2007)
# define __attribute__(x)
#endif
[...]
#ifndef ATTRIBUTE_UNUSED
#define ATTRIBUTE_UNUSED __attribute__ ((__unused__))
#endif /* ATTRIBUTE_UNUSED */
Should it have an ATTRIBUTE_ prefix, perhaps? Not sure. (but that's
heading into bikeshed territory; more important to actually get it
working)
David> +#else
David> + #define CPYCHECKER_RETURNS_BORROWED_REF()
David> + #error (This should have been defined)
David> +#endif
I think having a #error here is wrong.
It is ok not to use the plugin ;-)
Good catch - in case this wasn't obvious, I meant to remove this when I
copied and pasted from the code in the automated test case.
(Am working on more of these custom attributes for cpychecker)
Thanks!
Dave
7:52 p.m.
On Wed, 2011-10-26 at 14:40 -0400, David Malcolm wrote:
(Am working on more of these custom attributes for cpychecker)
I've added an attribute for marking functions that steal reference(s) to
their argument(s) in:
http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=commitdiff;h=e...
I then used this to mark a function within the plugin itself in:
http://git.fedorahosted.org/git/?p=gcc-python-plugin.git;a=commitdiff;h=1...
This marking successfully suppressed a lot of false warnings when
running the checker on itself.
Dave
9:04 a.m.
David> I was going to say "sure", but I'm a bit nervous about this - I
don't
David> want to get into the business of actually fixing gdb's bugs.
Yeah, I totally understand. I'll make a branch for the needed changes.
If you're still testing the plugin on gdb, I'll let you when I make it.
Tom> The gdb style is to use object-like, not function-like, macros for this
Tom> kind of thing.
David> By "object-like", do you mean without parentheses?
Yes.
David> Should it have an ATTRIBUTE_ prefix, perhaps? Not sure. (but that's
David> heading into bikeshed territory; more important to actually get it
David> working)
I don't really mind either way.
If these attributes end up in Python itself, and if Python exports them,
then I think we will just follow their naming convention in the interest
of clarity.
Tom
12:36 p.m.
Tom> If you're still testing the plugin on gdb, I'll let you when I make it.
I pushed archer-tromey-python-checker.
It is based on a recent gdb/master. I'll merge from there periodically.
All that is on the branch right now is a slightly modified version of
your attribute patch.
Tom
4566
days inactive
4568
days old
gcc-python-plugin@lists.fedorahosted.org
10 comments
3 participants
participants (3)
-
David Malcolm -
Mark Wielaard -
Tom Tromey