4:29 p.m.
Hi all,
First of all thank you for working on Fedora-Review, which was really useful
for me -- saved me a lot of time and probably improved quality of my reviews.
I would like to share my opinion on how Fedora-Review plugin API should look
like from perspective of a user who did a number of reviews and who doesn't
know the internal API.
PREAMBLE
Normal F-R workflow (simplified a lot): First you build the package and run a
number of automated tests (as part of F-R and external tools). Then go through
the review template and perform manual checks. In most cases you can't open and
read every source file, but you you'll probably end up running tools like find
and grep looking for some package-specific patterns.
After doing your n-th review you realize which parts are boring and brainless
-- you find candidates for test automation. At this point you could probably
write a custom script or a F-R plugin (using Python or JSON API).
Writing a custom plugin requires a lot of time (I know from my own experience),
so I think that the first option is definitely better. After all you remember
well all the commands you had to run, you just need to write them down in a
form of a script. You can even copy variuos parts directly from your shell
history. But running these scripts manually again and again can be
annoying. What you need is a glue between F-R and the scripts -- the API I am
proposing.
SOME ASSUMPTIONS
The API should:
1) Be easy to learn.
If you need to spend several (or more) hours to learn the F-R API and API of
JSON some JSON library then creating a custom test (plugin) may be not worth
the effort. However if you somehoe know most of the API before you even start
thinking about plugins (more about that below), creating test scripts will be
straightforward.
2) Be multilangual.
You may want to write the simplest tests in just Bourne shell, maybe more
complicated ones in other scripting languages. Also Perl SIG may want to write
tests in Perl, Python in Python, Java SIG in... anything but Java :) You can't
expect Perl proponents to write Python code only to create a plugin for F-R.
3) Backwards compatible.
If you spend some time developing a set of plugins you don't want to rewrite
them because of F-R upgrade. During upgrade from 0.1.x to 0.2.x I had to
rewrite my plugin. Hopefully I had only one and Stano helped me with the
upgrade.
4) Callback-less.
Current JSON API gives plugins only some information about package being
reviewed and requires them to call F-R back to obtain more info (like sections
of spec file). This means formating a request, reading and parsing
(semi-manually!) the reply. But this information could have been automatically
provided by F-R from the beginning! F-R should give pluging all info it has
about the package without plugins having to ask for it.
MY PROPOSAL
Basic information about packages, like name, version, type (C,Java,R,...) is
passed in exported environmental variables, which names should be compatible
with names from `rpm --querytags` if possible. More detailed information is
passed on disk in form of files. This includes things like spec file, unpacked
sources, rpmlint output, symlinks to builddir, unpacked binary rpms and so on.
Users already should know basic RPM macros like %{name} or %{epoch}, so using
env variables ($name and $epoch respectively) should be strightforward. People
that previously used F-R will be familiar with structure of directories created
by F-R (srpm-unpacked, upstream, BUILD, and so on). Test results are simply
printed on stdout. Exit code decides if test passed, failed or was inconclusive
(exit codes for that are already standardized). Attachements can be created by
creating attachement files somewhere in the filesystem.
I believe that implementing something similar to my approach would greatly
enchance Fedora Review. It could be implemented as a JSON plugin (what I
actually did) which then has a separate api for comunicating with different,
scripts or implemented directly in Python. It's up to you, Fedora-Review
developers, to decide if you want to support my idea or not. If yes then I
would be glad to help. In any case I have my implementation working now
so I'm happy either way :)
Keep up with good work!
Mikolaj Izdebski
7:05 a.m.
On 08/08/2012 11:29 PM, Mikolaj Izdebski wrote:
Hi all,
First of all thank you for working on Fedora-Review, which was really useful
for me -- saved me a lot of time and probably improved quality of my reviews.
I would like to share my opinion on how Fedora-Review plugin API should look
like from perspective of a user who did a number of reviews and who doesn't
know the internal API.
PREAMBLE
Normal F-R workflow (simplified a lot): First you build the package and run a
number of automated tests (as part of F-R and external tools). Then go through
the review template and perform manual checks. In most cases you can't open and
read every source file, but you you'll probably end up running tools like find
and grep looking for some package-specific patterns.
After doing your n-th review you realize which parts are boring and brainless
-- you find candidates for test automation. At this point you could probably
write a custom script or a F-R plugin (using Python or JSON API).
Writing a custom plugin requires a lot of time (I know from my own experience),
so I think that the first option is definitely better. After all you remember
well all the commands you had to run, you just need to write them down in a
form of a script. You can even copy variuos parts directly from your shell
history. But running these scripts manually again and again can be
annoying. What you need is a glue between F-R and the scripts -- the API I am
proposing.
SOME ASSUMPTIONS
The API should:
1) Be easy to learn.
If you need to spend several (or more) hours to learn the F-R API and API of
JSON some JSON library then creating a custom test (plugin) may be not worth
the effort. However if you somehoe know most of the API before you even start
thinking about plugins (more about that below), creating test scripts will be
straightforward.
2) Be multilangual.
You may want to write the simplest tests in just Bourne shell, maybe more
complicated ones in other scripting languages. Also Perl SIG may want to write
tests in Perl, Python in Python, Java SIG in... anything but Java :) You can't
expect Perl proponents to write Python code only to create a plugin for F-R.
3) Backwards compatible.
If you spend some time developing a set of plugins you don't want to rewrite
them because of F-R upgrade. During upgrade from 0.1.x to 0.2.x I had to
rewrite my plugin. Hopefully I had only one and Stano helped me with the
upgrade.
4) Callback-less.
Current JSON API gives plugins only some information about package being
reviewed and requires them to call F-R back to obtain more info (like sections
of spec file). This means formating a request, reading and parsing
(semi-manually!) the reply. But this information could have been automatically
provided by F-R from the beginning! F-R should give pluging all info it has
about the package without plugins having to ask for it.
MY PROPOSAL
Basic information about packages, like name, version, type (C,Java,R,...) is
passed in exported environmental variables, which names should be compatible
with names from `rpm --querytags` if possible. More detailed information is
passed on disk in form of files. This includes things like spec file, unpacked
sources, rpmlint output, symlinks to builddir, unpacked binary rpms and so on.
Users already should know basic RPM macros like %{name} or %{epoch}, so using
env variables ($name and $epoch respectively) should be strightforward. People
that previously used F-R will be familiar with structure of directories created
by F-R (srpm-unpacked, upstream, BUILD, and so on). Test results are simply
printed on stdout. Exit code decides if test passed, failed or was inconclusive
(exit codes for that are already standardized). Attachements can be created by
creating attachement files somewhere in the filesystem.
[cut]
First of all: thanks for bringing up this issue!
Agreed: Easy to learn, mutlilingual, backwards compatible, no callbacks:
Considerations.
As noted in #96, the framework need th plugin to register it's tests.
Limiting the plugins to interpreted langauges, this could be as simple
as some greppable comments about name, info, version for each test.
The role of JSON: F-r will just start the plugin with a list of tests to
run. No json is needed for this, it only requires simple command line
parameters. Json is probably the right way to return data, though. So, a
plugin don't need to decode json, just to encode the reply.
The info: as of now, f-r already runs within a review directory with
fixed names. So, a plugin will find everything like sources, build
direcory, unpacked srpm etc at fixed paths from current directory. There
are some exceptions (notably rpmlint output), but that should be easy to
add.
However, I'm not enterily happy with the "everything in environment
variables" approach. If there is no way for the plugin to define what
it wants (parameters), the interface will expand. Just documenting it
would be ... almost wrote 'a nightmare' . Oh, did I ?
I think one thing which could work is to create Yet Another Tool (yat).
yat would just used to grab things from the review directory, doing
things like returning spec sections, built rpms etc. The interface to
this tool could be much smaller than providing everything in variables,
and also more flexible (supporting e. g, get_files_by_pattern() etc). It
could be built from existing f-r classes, with a new interface.
Summing up: a plugin sketch:
- Plugins are interpreted (have comments), not compiled.
- Comments for registration: What tests can I run, name, version, info...
- Invoked with a list of tests to run + options for debug etc. Simple
CLI interface.
- Environment variable or simple config file describes basic stuff:
type, version, name etc.
- Invoked in review dir, with all reasonable files at fixed locations
- Can (but don't need to) use the yat tool to access the data at higher
level.
- Returns a json object as today, although it can also be an error object.
7:24 a.m.
However, I'm not enterily happy with the "everything in
environment
variables" approach. If there is no way for the plugin to define what
it wants (parameters), the interface will expand. Just documenting it
would be ... almost wrote 'a nightmare' . Oh, did I ?
I never said "everything". Only the most basic information: like name,
version, release, epoch. 10 variables tops, with simple contents.
- Plugins are interpreted (have comments), not compiled.
A compiled plugin can also embed some info. Like in C/C++:
static const char *info = "# Name: foo\nGroup: Generic\n";
The same can be done for other compiled languages too.
- Comments for registration: What tests can I run, name, version,
info...
Comments should be optional and have some default sane values, like:
* name (default: file name),
* group (default: Generic),
* URL of guidelines (default: no URL)
* if it's a must or should check (default: should),
* list of tests it deprecates (default: empty list)
* dependencies between tests (optionally, if F-R supports them)
- Invoked with a list of tests to run + options for debug etc. Simple
CLI interface.
I assumed there would be only 1 test per file, but that could be xtended of course.
- Returns a json object as today, although it can also be an error
object.
I really don't like this point. Usually there are only 2 things to be returned:
check status and additional notes. IMO the first one should just be exit status
of the process, the second - stdout. If there are some needs, you could alternatively
return a JSON object. So you first try to parse the reply as JSON, if it fails you
assume it's plain text.
Mikolaj
7:58 a.m.
On 08/09/2012 02:24 PM, Mikolaj Izdebski wrote:
CLI interface.
I assumed there would be only 1 test per file, but that could be xtended of course.
It might make sense to have several tests in on file IMHO. BTW, the
current json return value is actually a list of results for that reason.
> - Returns a json object as today, although it can also be an error object.
I really don't like this point. Usually there are only 2 things to be returned:
check status and additional notes. IMO the first one should just be exit status
of the process, the second - stdout. If there are some needs, you could alternatively
return a JSON object. So you first try to parse the reply as JSON, if it fails you
assume it's plain text.
Niote that the current version also returns attachments (this is not
implemented in the current plugin interface :( ) So, we are returning a
list of results. Each result contains a list of attachments. So. some
kind of format is needed. We can invent our own, use e. g. YAML or just
stick with json. Being lazy, I prefer the last...
That said, simple returns like status+ notes could be returned as
lines, one per test.
BTW,, an alternative might be to skip the comments and expand the cli
interface to accept two commands: 'config' which returns registration
data, and 'run <tests>' which runs the tests. Not much parsing is needed
to handle that...
8:13 a.m.
I just want to make sure that simple things are simple, like:
#!/bin/sh
# @Group: Java
# @Deprecates: CheckNoArch
# @Description: Package has BuildArch: noarch (if possible)
egrep -q -i '^ *BuildArch: *noarch *$' srpm-unpacked/$name.spec
(Note the use of $name
Or:
#!/bin/sh
# @Deprecates: CheckLargeDocs
# @Strict
# @URL: http://fedoraproject.org/wiki/Packaging/Guidelines#PackageDocumentation
# @Description: Large documentation files must go in a -doc subpackage.
min=10000
max=1000000
docsize=$(for rpm in ./rpms/*; do (cd ./$rpm; test -d ./usr/share/doc && tar c
./usr/share/doc); done | wc -c)
echo Documentation size is $docsize bytes.
test $docsize -lt $min && exit 0
test $docsize -gt $max && exit 2
exit 1
That tests are very simple and yet make my life easier. Using JSON for this
is definitely an overkill and contradicts the "should be easy to learn" goal.
8:22 a.m.
On 08/09/2012 03:13 PM, Mikolaj Izdebski wrote:
I just want to make sure that simple things are simple, like:
#!/bin/sh
# @Group: Java
# @Deprecates: CheckNoArch
# @Description: Package has BuildArch: noarch (if possible)
egrep -q -i '^ *BuildArch: *noarch *$' srpm-unpacked/$name.spec
(Note the use of $name
Or:
#!/bin/sh
# @Deprecates: CheckLargeDocs
# @Strict
# @URL: http://fedoraproject.org/wiki/Packaging/Guidelines#PackageDocumentation
# @Description: Large documentation files must go in a -doc subpackage.
min=10000
max=1000000
docsize=$(for rpm in ./rpms/*; do (cd ./$rpm; test -d ./usr/share/doc && tar c
./usr/share/doc); done | wc -c)
echo Documentation size is $docsize bytes.
test $docsize -lt $min && exit 0
test $docsize -gt $max && exit 2
exit 1
That tests are very simple and yet make my life easier. Using JSON for this
is definitely an overkill and contradicts the "should be easy to learn" goal.
_______________________________________________
fedorareview mailing list
fedorareview(a)lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/fedorareview
Examples are good,
especially examples like these. To be fair, I hadn't
realized how simple things could be. Have you implemented this using a
json plugin handling the communication between "your" plugins and f-r?!
--alec
8:29 a.m.
Examples are good, especially examples like these. To be fair, I
hadn't
realized how simple things could be. Have you implemented this using a
json plugin handling the communication between "your" plugins and f-r?!
Yes, I have implemented this as a single JSON plugin which basically:
1) asks F-R for all possible info (spec file scetions etc)
2) extracts more stuff on disk
3) reads metadata of all scripts
4) orders scripts basing on their dependencies
5) executes individual scripts
6) returns one reply to F-R containing results of all tests
I would love to release this code to the public, but I need to ask my
employer for consent before doing so.
9:16 a.m.
On 08/09/2012 03:29 PM, Mikolaj Izdebski wrote:
> Examples are good, especially examples like these. To be fair, I
hadn't
> realized how simple things could be. Have you implemented this using a
> json plugin handling the communication between "your" plugins and f-r?!
Yes, I have implemented this as a single JSON plugin which basically:
1) asks F-R for all possible info (spec file scetions etc)
2) extracts more stuff on disk
3) reads metadata of all scripts
4) orders scripts basing on their dependencies
5) executes individual scripts
6) returns one reply to F-R containing results of all tests
I raised the issue to simply drop the json api in another message. One
strategy might be to support either "advanced" plugins written in
python or "simple" ones following your approach here. This means that if
we could reimplement your json plugin in python the json interface could
be dropped and we would have:
- python plugins: multiple tests, attachment handling, access to
internal python classes.
- "simple" plugins: one test per file, simplified registration, no
attachments.
To me, this looks attractive. Let's hope your employer sees the benefits
in opening up the sources.
In any case, the modelling of a test needs an overhaul both in python
and to some extent also for the simple ones. Things like
dependencies/execution order, access to other tests, selecting tests to
run. The registration of plugin tests should really be done also by tje
python ones...
--alec
10:01 a.m.
Quoting Alec Leamas (2012-08-09 16:16:02)
On 08/09/2012 03:29 PM, Mikolaj Izdebski wrote:
>> Examples are good, especially examples like these. To be fair, I hadn't
>> realized how simple things could be. Have you implemented this using a
>> json plugin handling the communication between "your" plugins and
f-r?!
> Yes, I have implemented this as a single JSON plugin which basically:
> 1) asks F-R for all possible info (spec file scetions etc)
> 2) extracts more stuff on disk
> 3) reads metadata of all scripts
> 4) orders scripts basing on their dependencies
> 5) executes individual scripts
> 6) returns one reply to F-R containing results of all tests
>
I raised the issue to simply drop the json api in another message. One
strategy might be to support either "advanced" plugins written in
python or "simple" ones following your approach here. This means that if
we could reimplement your json plugin in python the json interface could
be dropped and we would have:
- python plugins: multiple tests, attachment handling, access to
internal python classes.
- "simple" plugins: one test per file, simplified registration, no
I wanted to have external tests on par with internal (Python) ones so
noone would feel left out. But it is most probably true that covering
90% functionality in external tests in more simple way would be more
productive (and attractive).
In any case, the modelling of a test needs an overhaul both in
python
and to some extent also for the simple ones. Things like
dependencies/execution order, access to other tests, selecting tests to
run. The registration of plugin tests should really be done also by tje
python ones...
Indeed my biggest grief is that internal tests have no idea about
external ones (from deprecating POV). For this we'd need 2-step plugin
interaction (i.e. registration and then run).
So alternative approach. Let's reimplement Mikolaj's interface in Python
where we put stuff either in ENV or special predefined places (i.e.
spec/main, spec/prep, spec/build, spec/install etc). And then get
information about plugins from decorators (name, description text etc).
Attachments can be put in special subdirectory (as Mikolaj suggested).
Normally they are supposed to be linked with specific test, not sure if
it's really that needed. Result of test is based on return value (0 -
success, 1 - fail, etc.).
Sounds like a plan?
--
Stanislav Ochotnicky <sochotnicky(a)redhat.com>
Software Engineer - Base Operating Systems Brno
PGP: 7B087241
Red Hat Inc. http://cz.redhat.com
11:15 a.m.
On 08/09/2012 05:01 PM, Stanislav Ochotnicky wrote:
Quoting Alec Leamas (2012-08-09 16:16:02)
> On 08/09/2012 03:29 PM, Mikolaj Izdebski wrote:
>>> Examples are good, especially examples like these. To be fair, I hadn't
>>> realized how simple things could be. Have you implemented this using a
>>> json plugin handling the communication between "your" plugins and
f-r?!
>> Yes, I have implemented this as a single JSON plugin which basically:
>> 1) asks F-R for all possible info (spec file scetions etc)
>> 2) extracts more stuff on disk
>> 3) reads metadata of all scripts
>> 4) orders scripts basing on their dependencies
>> 5) executes individual scripts
>> 6) returns one reply to F-R containing results of all tests
>>
> I raised the issue to simply drop the json api in another message. One
> strategy might be to support either "advanced" plugins written in
> python or "simple" ones following your approach here. This means that if
> we could reimplement your json plugin in python the json interface could
> be dropped and we would have:
> - python plugins: multiple tests, attachment handling, access to
> internal python classes.
> - "simple" plugins: one test per file, simplified registration, no
I wanted to have external tests on par with internal (Python) ones so
noone would feel left out. But it is most probably true that covering
90% functionality in external tests in more simple way would be more
productive (and attractive).
> In any case, the modelling of a test needs an overhaul both in python
> and to some extent also for the simple ones. Things like
> dependencies/execution order, access to other tests, selecting tests to
> run. The registration of plugin tests should really be done also by tje
> python ones...
Indeed my biggest grief is that internal tests have no idea about
external ones (from deprecating POV). For this we'd need 2-step plugin
interaction (i.e. registration and then run).
So alternative approach. Let's reimplement Mikolaj's interface in Python
where we put stuff either in ENV or special predefined places (i.e.
spec/main, spec/prep, spec/build, spec/install etc). And then get
information about plugins from decorators (name, description text etc).
Attachments can be put in special subdirectory (as Mikolaj suggested).
Normally they are supposed to be linked with specific test, not sure if
it's really that needed. Result of test is based on return value (0 -
success, 1 - fail, etc.).
Sounds like a plan?
yes, it sounds like a plan. And a good one.
But as you wrote earlier, the registration needs a two-phase approach:
first register all tests (internal and external), then run. I don't
think we save time by cutting this corner - I think we need to do that
first. In such a model, internal and external tests should be equals.
after registration.
I'm still not happy with e. g. splitting spec into sections, having each
in a file as the only way. This path will eventually expand the
interface beyond sanity, when f-r should give all possible information
in advance for any possible need.. I still think the yat approach, a
tool which can extract info from the files on behalf of the plugin is a
better one (combined with the raw data in the review directory.)
Basically, yat would provide similar info as python plugins can retrieve
from their context. It could also easily filter the output depending on
settings, an otherwise complicated task.
This is not black or white. Things which are needed "often" qualify for
a file;others are better accessed using yat. It's a trade-off between
simple interfaces and keeping them flexible and limited in size.
So, my two points
- First create a sane model for tests including registration,
dependecies, access to results.
- Augment the files interface with a tool capable of more high-level
access.
3:49 p.m.
On 08/09/2012 05:01 PM, Stanislav Ochotnicky wrote:
Quoting Alec Leamas (2012-08-09 16:16:02)
> On 08/09/2012 03:29 PM, Mikolaj Izdebski wrote:
>>> Examples are good, especially examples like these. To be fair, I hadn't
>>> realized how simple things could be. Have you implemented this using a
>>> json plugin handling the communication between "your" plugins and
f-r?!
>> Yes, I have implemented this as a single JSON plugin which basically:
>> 1) asks F-R for all possible info (spec file scetions etc)
>> 2) extracts more stuff on disk
>> 3) reads metadata of all scripts
>> 4) orders scripts basing on their dependencies
>> 5) executes individual scripts
>> 6) returns one reply to F-R containing results of all tests
>>
> I raised the issue to simply drop the json api in another message. One
> strategy might be to support either "advanced" plugins written in
> python or "simple" ones following your approach here. This means that if
> we could reimplement your json plugin in python the json interface could
> be dropped and we would have:
> - python plugins: multiple tests, attachment handling, access to
> internal python classes.
> - "simple" plugins: one test per file, simplified registration, no
I wanted to have external tests on par with internal (Python) ones so
noone would feel left out. But it is most probably true that covering
90% functionality in external tests in more simple way would be more
productive (and attractive).
> In any case, the modelling of a test needs an overhaul both in python
> and to some extent also for the simple ones. Things like
> dependencies/execution order, access to other tests, selecting tests to
> run. The registration of plugin tests should really be done also by tje
> python ones...
Indeed my biggest grief is that internal tests have no idea about
external ones (from deprecating POV). For this we'd need 2-step plugin
interaction (i.e. registration and then run).
So alternative approach. Let's reimplement Mikolaj's interface in Python
where we put stuff either in ENV or special predefined places (i.e.
spec/main, spec/prep, spec/build, spec/install etc). And then get
information about plugins from decorators (name, description text etc).
Attachments can be put in special subdirectory (as Mikolaj suggested).
Normally they are supposed to be linked with specific test, not sure if
it's really that needed. Result of test is based on return value (0 -
success, 1 - fail, etc.).
Sounds like a plan?
Yes, it's a plan....(#2)
I have pushed a feature branch check-api which handles the problems I
have seen from the tool's point of view. The most important part is that
plugins is registered so user can list, select etc. among the defined
tests. Also, to get a reasonable code all tests, internal or external,
is represented in the same way and stored in a common place.
The rest step depends, if I understand it right, on Mikolaj's employer
and whether he (she?) is willing to publish his work under an open
license. Note that rewriting the json plugin really don't requires the
code, but rather the environment expected by the shell tools. If
Mikolaj could define this, I dont think writing a python "bridge"
plugin is much work.:
9:47 a.m.
Quoting Mikolaj Izdebski (2012-08-09 15:29:28)
> Examples are good, especially examples like these. To be fair, I
hadn't
> realized how simple things could be. Have you implemented this using a
> json plugin handling the communication between "your" plugins and f-r?!
Yes, I have implemented this as a single JSON plugin which basically:
1) asks F-R for all possible info (spec file scetions etc)
2) extracts more stuff on disk
3) reads metadata of all scripts
4) orders scripts basing on their dependencies
5) executes individual scripts
6) returns one reply to F-R containing results of all tests
I would love to release this code to the public, but I need to ask my
employer for consent before doing so.
And I see no reason why we couldn't add this JSON plugin directly into
f-r so that people can write these simple tests in shell (or whatever
decorated madness). We can install it by default into plugin directory,
and
JSONPlugin *almost* works (proved by Mikolaj being able to create
additional checks with it :-) ).
What is missing/wrong:
* Callback nature is sub-optimal. We can extract a lot of stuff and
place it in specific subdirectories in review dir similar to how
Mikolaj's plugins is already doing. We just do it automatically for
everyone (even internal plugins if they are so inclined).
* My original idea was that each group interested in certain package
type would create their "base" check and normal checks would be based
on that. So they would need to develop the JSON handling only once.
I'd even be inclined to package such wrappers directly in f-r to ease
stuff. It was not meant for each reviewer to create their own private
set of checks. And I still don't think we should be aiming there
primarily
So action items:
* Add basic information into ENV variables before calling plugins
* Add support for attachments (maybe I'll just scan contents of certain
fixed directory and consider each file as an attachment?)
* Create basic plugins in Perl/Ruby/whatever and install them in system
directories so tests can use them as their base tests in more simple
way without caring about JSON stuff. Plus we'll keep them updated
together with API if needed. We already have basic Perl stuff in the
example, just need to make it more modular so it can be reused.
* Include Mikolaj's plugin so people can create *really* simple stuff
with small shell scripts
Or does it sound really that bad?
--
Stanislav Ochotnicky <sochotnicky(a)redhat.com>
Software Engineer - Base Operating Systems Brno
PGP: 7B087241
Red Hat Inc. http://cz.redhat.com
11:28 a.m.
On 08/09/2012 04:47 PM, Stanislav Ochotnicky wrote:
Quoting Mikolaj Izdebski (2012-08-09 15:29:28)
>> Examples are good, especially examples like these. To be fair, I hadn't
>> realized how simple things could be. Have you implemented this using a
>> json plugin handling the communication between "your" plugins and
f-r?!
> Yes, I have implemented this as a single JSON plugin which basically:
> 1) asks F-R for all possible info (spec file scetions etc)
> 2) extracts more stuff on disk
> 3) reads metadata of all scripts
> 4) orders scripts basing on their dependencies
> 5) executes individual scripts
> 6) returns one reply to F-R containing results of all tests
>
> I would love to release this code to the public, but I need to ask my
> employer for consent before doing so.
And I see no reason why we couldn't add this JSON plugin directly into
f-r so that people can write these simple tests in shell (or whatever
decorated madness). We can install it by default into plugin directory,
and
JSONPlugin *almost* works (proved by Mikolaj being able to create
additional checks with it :-) ).
What is missing/wrong:
* Callback nature is sub-optimal. We can extract a lot of stuff and
place it in specific subdirectories in review dir similar to how
Mikolaj's plugins is already doing. We just do it automatically for
everyone (even internal plugins if they are so inclined).
* My original idea was that each group interested in certain package
type would create their "base" check and normal checks would be based
on that. So they would need to develop the JSON handling only once.
I'd even be inclined to package such wrappers directly in f-r to ease
stuff. It was not meant for each reviewer to create their own private
set of checks. And I still don't think we should be aiming there
primarily
So action items:
* Add basic information into ENV variables before calling plugins
Can't
hurt...
* Add support for attachments (maybe I'll just scan contents of
certain
fixed directory and consider each file as an attachment?)
Problem is that json
interface needs to be fixed dirst, certainly doable.
* Create basic plugins in Perl/Ruby/whatever and install them in
system
directories so tests can use them as their base tests in more simple
way without caring about JSON stuff. Plus we'll keep them updated
together with API if needed. We already have basic Perl stuff in the
example, just need to make it more modular so it can be reused.
Which is fine,
only we get the registration data right. We don't want to
change that API from day 1, do we? Also note that running through json
means that we must expand that interface to transfer registtration data
from the simple plugins over Mikolaj's plugin over JSON to f-r (besides
attachments)
Maybe we would be better off rethinking the registration first so we can
make a reasonable attempt on the plugin registration data? And skipping
attachments for simple plugins until we have reimplemented MIkolaj's
plugin in python? instead of wasting time on the json interface?
* Include Mikolaj's plugin so people can create *really* simple
stuff
with small shell scripts
Or does it sound really that bad?
No, not that bad. The yat tool thing can be posponed, and evolve over
time. But I would sure like to get things right first: registration, API...
--alec
7:53 a.m.
New subject: New plugin API (long)
Top-posting, have mercy...
I have *a sketch* to a python plugin which can handle the two examples
below. I have tried to do it in a way where adding new features should
be easy. While keeping the overall ideas, I have messed with some
details. My variant of check-large docs should demonstrate some subtle
changes:
#!/bin/bash
# @deprecates: CheckLargeDocs
# @group: Generic
# @url:
http://fedoraproject.org/wiki/Packaging/Guidelines#PackageDocumentation
# @text: Large documentation files must go in a -doc subpackage.
unpack_rpms
min=10000
max=1000000
docsize=$(for rpm in rpms-unpacked/*; do
(cd $rpm; test -d ./usr/share/doc && tar c ./usr/share/doc)
done | wc -c)
echo Documentation size is $docsize bytes.
test $docsize -lt $min && exit $FR_PASS
test $docsize -gt $max && exit $FR_FAIL
exit $FR_PENDING
Changes and possible discussions:
- Instead of defining environment variables when calling the script,
I write a bash file which is sourced before calling. This means that not
only plain, scalar variables are available. The script contains e. g.
the function unpack_rpms used here. I have just been involved in a 1 Gb
package, and I wouldn't want that unpacked without purpose...
- Instead of splitting the spec into separate files I use bash
associative arrays (i. e., hashes) to publish spec data
- I'm not really happy with using return codes like 1, 2 and three for
regular returns. It makes it impossible to detect the errors which will
occur. So, some symbolic constants like here might be good (defined in
the span 80-90) Using 0 for 'pass' is certainly possible anyway.
- Declaring the "Group" tag means that this test wilil only be invoked
if it's applicable for this package. So, there must already be existing
python tests for this group. You cannot define a new group using this API.
I attach the generated file sourced before running scripts; it's somehow
the framework of the interface.
On 08/09/2012 03:13 PM, Mikolaj Izdebski wrote:
I just want to make sure that simple things are simple, like:
#!/bin/sh
# @Group: Java
# @Deprecates: CheckNoArch
# @Description: Package has BuildArch: noarch (if possible)
egrep -q -i '^ *BuildArch: *noarch *$' srpm-unpacked/$name.spec
(Note the use of $name
Or:
#!/bin/sh
# @Deprecates: CheckLargeDocs
# @Strict
# @URL: http://fedoraproject.org/wiki/Packaging/Guidelines#PackageDocumentation
# @Description: Large documentation files must go in a -doc subpackage.
min=10000
max=1000000
docsize=$(for rpm in ./rpms/*; do (cd ./$rpm; test -d ./usr/share/doc && tar c
./usr/share/doc); done | wc -c)
echo Documentation size is $docsize bytes.
test $docsize -lt $min && exit 0
test $docsize -gt $max && exit 2
exit 1
That tests are very simple and yet make my life easier. Using JSON for this
is definitely an overkill and contradicts the "should be easy to learn" goal.
_______________________________________________
fedorareview mailing list
fedorareview(a)lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/fedorareview
4271
days inactive
4279
days old
fedorareview@lists.fedorahosted.org
14 comments
3 participants
participants (3)
-
Alec Leamas -
Mikolaj Izdebski -
Stanislav Ochotnicky