> Forwarding to fedora-infrastructure-list soit canget more exposure and > discussion. > > -------- Original Message -------- > Subject: Tosca widgets, only half the battle > Date: Sun, 11 May 2008 12:27:36 -0400 > From: John (J5) Palmieri <johnp(a)redhat.com&gt; > To: Toshio Kuratomi <a.badger(a)gmail.com&gt; > CC: tcallawa(a)redhat.com, lmacken(a)redhat.com, mmcgrath(a)redhat.com > > After hacking away at MyFedora and producing a lot of ugly code in the > process I finally sat down the last two weeks to organize everything > into a framework make it much more extensible and have patterns for > people to easily create content. Most of the technologies are > solidifying into my head and I have been working on hashing out an API > design behind the user interaction design I had started with. The issue > I am running into now is the fact that Turbo Gears and related > technology come from a monolithic design and adhere too stringently to > the Model/View/Controller design pattern. This is really an issue when > your models, views and controllers can come from different applications > or even different servers. MyFedora is of course a mashup of different > tools and does not fit the, I'm grabbing data from a single database and > displaying it via a self contained template, mold. What I need is a > complete plugin system where a person can write their own self contained > controllers, templates and static files which then drop in and are > loaded on the fly, while integrating with the global project. > Do we want the myfedora app to be coded in such a way that it works with lots of technologies? or do we want to define a standard that the technologies can implement to make it work with myfedora?

On Wed, 2008-05-14 at 12:41 -0400, Luke Macken wrote: > On Wed, May 14, 2008 at 11:03:38AM -0500, Mike McGrath wrote: > > > > > Forwarding to fedora-infrastructure-list soit canget more exposure and > > > discussion. > > > > > > -------- Original Message -------- > > > Subject: Tosca widgets, only half the battle > > > Date: Sun, 11 May 2008 12:27:36 -0400 > > > From: John (J5) Palmieri <johnp(a)redhat.com&gt; > > > To: Toshio Kuratomi <a.badger(a)gmail.com&gt; > > > CC: tcallawa(a)redhat.com, lmacken(a)redhat.com, mmcgrath(a)redhat.com > > > > > > After hacking away at MyFedora and producing a lot of ugly code in the > > > process I finally sat down the last two weeks to organize everything > > > into a framework make it much more extensible and have patterns for > > > people to easily create content. Most of the technologies are > > > solidifying into my head and I have been working on hashing out an API > > > design behind the user interaction design I had started with. The issue > > > I am running into now is the fact that Turbo Gears and related > > > technology come from a monolithic design and adhere too stringently to > > > the Model/View/Controller design pattern. This is really an issue when > > > your models, views and controllers can come from different applications > > > or even different servers. MyFedora is of course a mashup of different > > > tools and does not fit the, I'm grabbing data from a single database and > > > displaying it via a self contained template, mold. What I need is a > > > complete plugin system where a person can write their own self contained > > > controllers, templates and static files which then drop in and are > > > loaded on the fly, while integrating with the global project. > > > > > > > Do we want the myfedora app to be coded in such a way that it works with > > lots of technologies? or do we want to define a standard that the > > technologies can implement to make it work with myfedora? > > I'd like to see us re-use and be compatible with as many existing > technologies and standards as possible. I don't necessarily see any > value in re-inventing our own. That is, unless we have a sound reason > to? > > luke Exactly, I would want a standard way of doing things at least in the MyFedora plugin context. This is why I didn't want to get away from templates and controllers (or something else that will be standard for a long time). Ideally someone who knows how to write a TG app should be able to write a plugin with a little glue code added and files shuffled around.

On Wed, May 14, 2008 at 09:06:24AM -0700, Toshio Kuratomi wrote: > Forwarding to fedora-infrastructure-list soit canget more exposure and > discussion. > > -------- Original Message -------- > Subject: Tosca widgets, only half the battle > Date: Sun, 11 May 2008 12:27:36 -0400 > From: John (J5) Palmieri <johnp(a)redhat.com&gt; > To: Toshio Kuratomi <a.badger(a)gmail.com&gt; > CC: tcallawa(a)redhat.com, lmacken(a)redhat.com, mmcgrath(a)redhat.com > > After hacking away at MyFedora and producing a lot of ugly code in the > process I finally sat down the last two weeks to organize everything > into a framework make it much more extensible and have patterns for > people to easily create content. Most of the technologies are > solidifying into my head and I have been working on hashing out an API > design behind the user interaction design I had started with. The issue > I am running into now is the fact that Turbo Gears and related > technology come from a monolithic design and adhere too stringently to > the Model/View/Controller design pattern. This is really an issue when > your models, views and controllers can come from different applications > or even different servers. MyFedora is of course a mashup of different > tools and does not fit the, I'm grabbing data from a single database and > displaying it via a self contained template, mold. What I need is a > complete plugin system where a person can write their own self contained > controllers, templates and static files which then drop in and are > loaded on the fly, while integrating with the global project. > > Before I go further let me describe my design. > > Vocabulary: > > Resource - This is the starting point for MyFedora plugins. A resource > is any abstract grouping such as "packages", "people" and "projects" > which contain tools for viewing and manipulating data within the > resource's context. > > Tools - A tool is a web app for viewing or manipulating data. For > example Builds would be a tool for the package resource. > > Data Id - The data id is a pointer to a specific dataset the tools work > on. For example the package resource considers each fedora package name > to be a data_id. > > The way things work are Resources are placed in the resources/ directory > and contain the logic for routing requests to a specific tool. They > also contain the master template which is a cause of path problems with > the current TG setup (include paths are relative to the including > template) > > Tools are placed in the tools/ directory and are controllers just like > any other TG controller. The exception is there is a standard for > including the master template and the tool pulls templates and static > files from its own directory. Tools can register with more than one > resource and must modify its behavior based on the resource calling it. > For instance the Build tool would be able to register with the package > and people resource and depending which resource is being used it would > display either a specific person's builds or the build history of a > package. Based on the resource being used the master template is pulled > in by the tool's templates. > > Data id's are simply what the resource passes to the tool and the tool > needs to be able to accept when dealing with a particular resource. For > instance the Packages resource would send a package name as a data id > and the Peoples resource would send a person's FAS username. > > The issue here is I need the tools to be self contained but still > integrate correctly with the global assests such as master templates and > graphics. Tosca widgets seemed to be the answer until I looked further > and found out they are just a higher level display layer than a self > contained controller/template system. It seems to be confusing because > it breaks the connection between the controller, data and the display > when I want that all to be encapsulated. Basically I don't want the > master page dolling out the data because the master page is just a > container to display the tool and links to other tools. The tools > should know where to get their data from. > > One solution is to use ToscaWidgets as a replacement for templates (or > more apt another layer between the controller and the template). That > makes things more complicated and throws away a lot of the concepts of > TG controllers. I guess I am probably just hung up on how I first > learned TG and we can just document around those issues. But another > thing to think about is stuff like WSGI. > > What do you guys think? Given my design and goals such as the ability to > display tools on the portal page, what is our plan of attack? How do we > concoct a plugin system to make it easy for others to create integrated > content while really just concentrating on their bits and not the wider > integration infrastructure? Are there systems/libraries out there that > already do this? Tosca is only part of the solution because it only > deals with encapsulating display and is mostly geared to generic widgets > like lists and not complete pages. I would like to have a framework > that is simple, focused and easy to use. You mention that you're running into issues with TurboGears v1.0's "monolithic design", which makes various assumptions about how your application is going to be implemented. I agree that this may not be ideal for the MyFedora environment, which is why I think basing this on TurboGears v2.0 is the way to go. Being built on top of Pylons, which does not make these strict assumptions, it will allow us to have much more control over how our application stack is structured. Since TG2/Pylons leverages the WSGI standard (which we are starting to adopt in our infrastructure deployment already), it will allow us to change any part of our own stack at any time, without taking apart the whole framework. If we treat resources as WSGI applications/middleware, it will make it trivial to plug these into our framework, package them, and develop them in isolation. ToscaWidgets is extremely useful, but not at this low of a level in our framework design. I think that TW will be great for the high-level re-usable widgets that can potentially use various MyFedora resources/tools, but for core MyFedora resources, I think basing them on top of the WSGI standard will be the most flexible solution.

On Thu, May 15, 2008 at 11:24:59AM -0400, John (J5) Palmieri wrote: > On Wed, 2008-05-14 at 12:39 -0400, Luke Macken wrote: > > On Wed, May 14, 2008 at 09:06:24AM -0700, Toshio Kuratomi wrote: > > > Forwarding to fedora-infrastructure-list soit canget more exposure and > > > discussion. > > > > > > -------- Original Message -------- > > > Subject: Tosca widgets, only half the battle > > > Date: Sun, 11 May 2008 12:27:36 -0400 > > > From: John (J5) Palmieri <johnp(a)redhat.com&gt; > > > To: Toshio Kuratomi <a.badger(a)gmail.com&gt; > > > CC: tcallawa(a)redhat.com, lmacken(a)redhat.com, mmcgrath(a)redhat.com > > > > > > After hacking away at MyFedora and producing a lot of ugly code in the > > > process I finally sat down the last two weeks to organize everything > > > into a framework make it much more extensible and have patterns for > > > people to easily create content. Most of the technologies are > > > solidifying into my head and I have been working on hashing out an API > > > design behind the user interaction design I had started with. The issue > > > I am running into now is the fact that Turbo Gears and related > > > technology come from a monolithic design and adhere too stringently to > > > the Model/View/Controller design pattern. This is really an issue when > > > your models, views and controllers can come from different applications > > > or even different servers. MyFedora is of course a mashup of different > > > tools and does not fit the, I'm grabbing data from a single database and > > > displaying it via a self contained template, mold. What I need is a > > > complete plugin system where a person can write their own self contained > > > controllers, templates and static files which then drop in and are > > > loaded on the fly, while integrating with the global project. > > > > > > Before I go further let me describe my design. > > > > > > Vocabulary: > > > > > > Resource - This is the starting point for MyFedora plugins. A resource > > > is any abstract grouping such as "packages", "people" and "projects" > > > which contain tools for viewing and manipulating data within the > > > resource's context. > > > > > > Tools - A tool is a web app for viewing or manipulating data. For > > > example Builds would be a tool for the package resource. > > > > > > Data Id - The data id is a pointer to a specific dataset the tools work > > > on. For example the package resource considers each fedora package name > > > to be a data_id. > > > > > > The way things work are Resources are placed in the resources/ directory > > > and contain the logic for routing requests to a specific tool. They > > > also contain the master template which is a cause of path problems with > > > the current TG setup (include paths are relative to the including > > > template) > > > > > > Tools are placed in the tools/ directory and are controllers just like > > > any other TG controller. The exception is there is a standard for > > > including the master template and the tool pulls templates and static > > > files from its own directory. Tools can register with more than one > > > resource and must modify its behavior based on the resource calling it. > > > For instance the Build tool would be able to register with the package > > > and people resource and depending which resource is being used it would > > > display either a specific person's builds or the build history of a > > > package. Based on the resource being used the master template is pulled > > > in by the tool's templates. > > > > > > Data id's are simply what the resource passes to the tool and the tool > > > needs to be able to accept when dealing with a particular resource. For > > > instance the Packages resource would send a package name as a data id > > > and the Peoples resource would send a person's FAS username. > > > > > > The issue here is I need the tools to be self contained but still > > > integrate correctly with the global assests such as master templates and > > > graphics. Tosca widgets seemed to be the answer until I looked further > > > and found out they are just a higher level display layer than a self > > > contained controller/template system. It seems to be confusing because > > > it breaks the connection between the controller, data and the display > > > when I want that all to be encapsulated. Basically I don't want the > > > master page dolling out the data because the master page is just a > > > container to display the tool and links to other tools. The tools > > > should know where to get their data from. > > > > > > One solution is to use ToscaWidgets as a replacement for templates (or > > > more apt another layer between the controller and the template). That > > > makes things more complicated and throws away a lot of the concepts of > > > TG controllers. I guess I am probably just hung up on how I first > > > learned TG and we can just document around those issues. But another > > > thing to think about is stuff like WSGI. > > > > > > What do you guys think? Given my design and goals such as the ability to > > > display tools on the portal page, what is our plan of attack? How do we > > > concoct a plugin system to make it easy for others to create integrated > > > content while really just concentrating on their bits and not the wider > > > integration infrastructure? Are there systems/libraries out there that > > > already do this? Tosca is only part of the solution because it only > > > deals with encapsulating display and is mostly geared to generic widgets > > > like lists and not complete pages. I would like to have a framework > > > that is simple, focused and easy to use. > > > > You mention that you're running into issues with TurboGears v1.0's > > "monolithic design", which makes various assumptions about how your > > application is going to be implemented. I agree that this may not be > > ideal for the MyFedora environment, which is why I think basing this on > > TurboGears v2.0 is the way to go. Being built on top of Pylons, which > > does not make these strict assumptions, it will allow us to have much > > more control over how our application stack is structured. Since > > TG2/Pylons leverages the WSGI standard (which we are starting to adopt > > in our infrastructure deployment already), it will allow us to change > > any part of our own stack at any time, without taking apart the whole > > framework. If we treat resources as WSGI applications/middleware, it > > will make it trivial to plug these into our framework, package them, and > > develop them in isolation. > > > > ToscaWidgets is extremely useful, but not at this low of a level in our > > framework design. I think that TW will be great for the high-level > > re-usable widgets that can potentially use various MyFedora > > resources/tools, but for core MyFedora resources, I think basing them on > > top of the WSGI standard will be the most flexible solution. > > I was thinking that but I am still not sure what WSGI is. Right now it > is just a magic term to me. Can you describe them in terms of > controllers right now? How does relative paths work (for instance with > template inheritance)? Is TG 2.0 stable enough for me to start rebasing > on? So, in terms of controller dispatching, they use two different mechanisms. TurboGears 1.0 uses CherryPy's object dispatch, which is essentially an attribute lookup for each path segment. Pylons uses Route's pattern matching against the full path to determine the controller to dispatch to. Basically, TurboGears object dispatching points to a function that is invoked by CherryPy, where Pylons points to a WSGI application. Since we're already using Routes for MyFedora, the switch to TG2 seems quite logical. TurboGears2 will use a more flexible object-dispatching system, and will also support direct Routes integration as well. For a great comparison of TurboGears (1.0) to Pylons, I recommend checking out this blog post by Ian Bicking: http://blog.ianbicking.org/turbogears-and-pylons.html luke _______________________________________________ Fedora-infrastructure-list mailing list Fedora-infrastructure-list(a)redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list

> > After hacking away at MyFedora and producing a lot of ugly code in the > process I finally sat down the last two weeks to organize everything > into a framework make it much more extensible and have patterns for > people to easily create content. Most of the technologies are > solidifying into my head and I have been working on hashing out an API > design behind the user interaction design I had started with. The issue > I am running into now is the fact that Turbo Gears and related > technology come from a monolithic design and adhere too stringently to > the Model/View/Controller design pattern. This is really an issue when > your models, views and controllers can come from different applications > or even different servers. MyFedora is of course a mashup of different > tools and does not fit the, I'm grabbing data from a single database and > displaying it via a self contained template, mold. What I need is a > complete plugin system where a person can write their own self contained > controllers, templates and static files which then drop in and are > loaded on the fly, while integrating with the global project. > FAS2 is at a similar stage. We want other people to be able to use FAS2. So we want to pull Fedora specific portions into a plugin architecture. We've started using setuptools entry-points since that's the plugin architecture that TurboGears already uses but that only solves how to load the modules, I think we got templates to work after a bit of hacking but I'd have to revisit the code. Getting the model to work is the big sticking point as the model means either: 1) A new db and a new set of db connections 2) Somehow merging the plugin's model with the main apps model. In myfedora, you might be able to use setuptools entry-points to better effect since you're operating largely without a model. If the plugins don't have to save state or can save very minimal state into a generic configs table you should be able to use entry-points to load up the code.

> Before I go further let me describe my design. > > Vocabulary: > > Resource - This is the starting point for MyFedora plugins. A resource > is any abstract grouping such as "packages", "people" and "projects" > which contain tools for viewing and manipulating data within the > resource's context. > > Tools - A tool is a web app for viewing or manipulating data. For > example Builds would be a tool for the package resource. > > Data Id - The data id is a pointer to a specific dataset the tools work > on. For example the package resource considers each fedora package name > to be a data_id. > > The way things work are Resources are placed in the resources/ directory > and contain the logic for routing requests to a specific tool. They > also contain the master template which is a cause of path problems with > the current TG setup (include paths are relative to the including > template) > This is how we fixed that in fas:: <?python import genshi.template.plugin master = genshi.template.plugin.MarkupTemplateEnginePlugin().load_template('fas.templates.master') ?> <xi:include href="${master.filename}" />

> Tools are placed in the tools/ directory and are controllers just like > any other TG controller. The exception is there is a standard for > including the master template and the tool pulls templates and static > files from its own directory. Tools can register with more than one > resource and must modify its behavior based on the resource calling it. > For instance the Build tool would be able to register with the package > and people resource and depending which resource is being used it would > display either a specific person's builds or the build history of a > package. Based on the resource being used the master template is pulled > in by the tool's templates. > That's pretty nifty. > Data id's are simply what the resource passes to the tool and the tool > needs to be able to accept when dealing with a particular resource. For > instance the Packages resource would send a package name as a data id > and the Peoples resource would send a person's FAS username. > An aside: It sounds like we'd want to implement some sort of global cache at some point. If we always start from the data_ids there's a lot of data to be pulled in each request. And if tools are using similar pieces of information but displaying it differently (for instance, when using a tool in a different resource) we'd definitely see reuse of the data.

> What do you guys think? Given my design and goals such as the ability to > display tools on the portal page, what is our plan of attack? How do we > concoct a plugin system to make it easy for others to create integrated > content while really just concentrating on their bits and not the wider > integration infrastructure? Are there systems/libraries out there that > already do this? Tosca is only part of the solution because it only > deals with encapsulating display and is mostly geared to generic widgets > like lists and not complete pages. I would like to have a framework > that is simple, focused and easy to use. > I suppose part of the question is who do we want to be building these plugins for MyFedora? If it's the author of the app providing the data then it should be something that their app can also use. If it's MyFedora developers, then it should be easy for them to create a new controller and setup a plugin. We might want to write a small script that makes a sample plugin directory like tg-admin quickstart does for the main app.

Take a look at how fas does things: fas/plugins/dummy_plugin. That should pretty much be a self-contained plugin using setuptools entrypoints to do its dirty work.