On 08/17/2018 04:18 PM, Michal Novotny wrote:>
Will the framework provide me with a way to automatically validate
against
a schema that I have locally defined e.g.
my_locally_defined_schema_that_i_can_then_publish_in_docs = {
"type" : "object",
"properties" : {
"price" : {"type" : "number"},
"name" : {"type" : "string"},
},}
if I want to use just the base Message? It would be good if I could
pass the body_scheme to Message __init__ method and have
that validated by api.publish, which would then invoke e.g.
message.validate().
Would it be possible?
If this all you want, just call ``jsonschema.validate`` yourself. Then
send it as a base Message.
Just be aware that
> any messages you send that way will integrate poorly (or not at all)
> with services like notifications.
>
I would like it to integrate well if possible...
Well... If FMN doesn't have a way to understand the messages, it can't
really do nice integration. I'm not sure what else to say here other
than if you want nice integration, you'll need to spend the ~15 minutes
required to publish your schema + minimal Python class.
Seriously, it's just:
$ python setup.py sdist bdist_wheel
$ twine upload <dists>
It's *really* easy. The time we've spent on this is vastly more than the
amount of work required.
>
> I really don't recommend this approach since we've been down this path
> before and it's worked rather poorly
Okay, but the question is why it was working poorly. From what I've seen,
the problems with fedmsg-meta would be just solved just by the explicit
scheme
validation on publisher side, which is really a cool thing you are
introducing
and it will help a lot.
Another thing that it is quite unclear why that service just got completely
stuck
when it couldn't process a message. Normally, you would just collect what
you
can collect out of the incoming message and send what you have collected
(i.e.
with some fields not filled). It only constructed human-readable strings
for notifications,
right? I don't think that's so mission critical even though important.
People do care about notifications some, but they become much less
useful if they're incomprehensibly JSON. FMN has plenty of problems
beyond the ever-changing messages that I won't burden you with, but yes,
a reasonable service handles bad input gracefully. This is why all
unparsable messages are now dropped by the notification service.
But anyway, still, if we have validation on publisher built-in into the
framework, that's
enough to solve that problem. Why would we need anything else?
Defining an API above a JSON object can be very useful. If you don't
need it, that's great, but it doesn't mean others don't want to use it.
> Schema are Immutable
> Message schema should be treated as immutable. Once defined, they should
not be altered. Instead, define a new schema class, mark the old one as
deprecated, and remove it after an appropriate transition period.
I think that adding new fields into the json should be simply allowed.
That's not a backward incompatible change. The
consumer doesn't use those fields because they have been just introduced so
you can't break a consuming script like that.
I agree the documentation here should be changed. Schema should not
change in a backwards-incompatible way, but additional keys in an object
are (by default) okay in JSON schema.
> Finally, you must distribute your schema to clients. It is recommended
that you maintain your message schema in your application’s git repository
in a separate Python package. The package name should be
<your-app-name>_schema.
What I was thinking about is to have the .json schema in a separate file
with recommendation that it contains URL to itself as ID (
https://pagure.io/copr/copr/raw/master/f/dist-git/my_schema.json). Given
that the schema can be self descriptive
(because there are the 'description' fields), then this should be
completely enough to even provide the documentation and at the same time,
that schema will be the actual schema publisher will use for validation.
I was looking into a Sphinx plugin to render documentation from the JSON
schema using its 'description' fields, so storing the docs in it make
sense. Storing them online is also useful for cross-language tooling and
is a feature of JSON schema.
I think this is much more chilled out approach which is, at the same time,
solving the problem with publisher sending something else than he/she
thinks he/she is sending. I think that was the main problem we were having,
no?
It's *a* problem we're having, but there are other problems as I've
enumerated in this thread. If you are not having those problems and
you're okay with minimal integration with other services like
notifications then there are innumerable ways to send messages. This
library is completely optional. You just need to have an AMQP client
library and add a few basic headers so clients using the
fedora-messaging library don't ignore your messages.
--
Jeremy Cline
XMPP: jeremy(a)jcline.org
IRC: jcline