17 posts tagged “catalyst”

I find as part of my normal course of writing perl moose based code, I do the following quite often:

Basically I tend to use the above with a core application class that is delegating work to several other classes.  In the above case, my general Library Class contains a Album class (presumable your library would hold an album and perhaps several other things).  In my Application logic I'm going to be calling methods on Album which have been properly normalized to the class which attempt to encapsulate the problem domain it attempts to solve.

I find the above form, although a bit verbose, to give me a lot of flexibility, particularly when I bring the application model into Catalyst via something like Catalyst::Model::Adaptor, which let's me define all the args in the global catalyst configuration.  Also, for testing, it's nice to be able to override the default classes, swapping in say a Mock Object or similar for the purposes of test cases.

In general use this construct commonly to avoid hardcoding all the connections between the elements of my applications.

Usage might be like:

Although I as mentioned, typically I instantiate this via Catalyst or as part of a more complicated application using an inversion of control container like Bread::Board. 

When I first started using Perl Catalyst for web application development I felt appalled by how sloppy it seemed to pass important application data around in a secondary global namespace, the Catalyst Stash.  I guess it's context global, but it stills seems messy to me in that it seems to poorly replicate features already built into Perl, and for which Perl can help you use correctly.  I mean, a global is bad enough, but if you use strict and warnings you will at least get warned if you abuse a global, such as if you try to use it before it's initialized and all that.  But the Stash is basically a Hash, and it's prone to typo issues, etc., basically all the evil stuff that I tried to banish from my Perl applications years ago.

It's one of those things that seems fundamentally busted to me, yet it works well enough for simple to medium cases.  It just feels to me like it decouples control way too much, and in my experience the stash becomes a sort of no man's land.

I think it's one of the three things that bother me about Catalyst (with the ->forward/go/visit/detach versus method call ambiguity and the way the default TT view guesses a template based on the action name being the other two things that work acceptably enough but still manage to bother me quite a bit).

Thoughts?  Am I being too controlling here?  Should I just go with the flow or are there others out there that are also bothered by this?

Introduction

This blog is a proposal and request for comments regarding adopting the XDG Filesystem Hierarchy as a option for managing all the non code data composing a Catalyst application.

The Problem

Right now when you create a new Catalyst application the non code data by default goes to either {home}/root (for templates and static stuff) or {home} (for configuration files), where {home} is the root directory of the application.  So you get a directory structure like:

MyApp
    myapp.conf
    /t
    /root
        /static
    /lib
        MyApp.pm
        /MyApp
            /Model
            /View
            /Controller

Now, this {home} directory is something of a hack, since we use Catalyst::Utils:home() to try to figure it out based on certain expectations.  Perl doesn't have this idea of {home} built into it.  If your application is 'installed' (via cpan or make install), we guess the location based on the physical address of the application modules (whatever you got that is inheriting from Catalyst).  If it's not installed (which is the common case when you are developing and just running the development server or tests) it walks the directory structure looking for a Makefile.PL or a Build.PL and then decides that's good enough to call {home}.

Since this method can be a bit flaky, a lot of people are recommending that you use File::ShareDir  (see here for a good overview).  This module intergrates well with Module::Install and leverages the fact that a Perl module can have a share directory associated with it.  Using this, you might create a directory structure like:

MyApp-Web
    /etc
        myapp-web.conf
    /t
    /share
        /static
    /lib
        MyApp.pm
        /MyApp
            Web.pm
            /Web
                /Model
                /View
                /Controller

I also modified the directory hierarchy a bit to reflect the growing consensus that your Catalyst application should ideally live one level further down from your application root.  In this case I choose 'MyApp/Web.pm' which seems to be the most popular choice and one that is semantically meaningful.  This represents the idea that your MVC layer should be the thinnest possible over your true domain and interface logic, which sits in the MyApp directory.  I also moved the configuration files to {home}/etc since that makes sense from people used to finding configuration in /etc

Although this is an improvement, it still suffers from several issues.  First of all one problem with File::ShareDir is that it can only find the share directory for installed applications.  For the common case where you are actively developing, or running tests, you still need some code like Catalyst::Utils::home() to guess the directory for you.  In this way it's not much better than what Catalyst::Utils::home() provides out of the box.

Also, when your share data is installed into the perl library path, this means that your application server (or user running apache mod_perl or fastcgi) would need the correct level of access to the path.  This complicates configuration.  This setup is this is not what most Unix administrators will expect.  There are reasonably well defined norms for where your configuration should go (/etc or ~/.config) as well as where the logs go and all that. 

Although you can override the {home} directory with environment variables, this is not ideal if our goal is to minimize installation hassle and make everything work well out of the box.  It complications your installation for users as well as configuration the web servers that will run the code.

It also complicate customization.  For example, let's say I am using the MojoMojo wiki and want to run three instance of it.  Each instance will have unique configuration and I want to slightly modify the theme files for each.  Right now, the only way I can do this is via the method of overridding the environment variable for home for each running instance.  Although this works, this is a 'roll my own' approach that is likely to vary from administrator to administrator, making it more difficult to onboard new admins due to the uniqueness of each application.  I strongly feel that we should have clear standards for all the most common case deployment issues, since this reduces errors, speeds deployment as well as counter the argument I often hear that Perl is hard to maintain.  A standard will also help grow a set of best practices surrounding deployment issues which we can document and promote.

Proposal

This is a case where Perl is not well leveraging existing norms, which really goes against the grain for us, considering CPAN with it's "reuse, recycle" mantra is one of our primary claims to fame.  My recommendation is that we adopt an existing standard and make this available as a plugin or set of roles for Catalyst.  The most relevent standard is the XDG Filesystem Hierarchy which exists specifically as a standard for where installed applications put configuration and data files, both locally that users can overide as well as global stuff that only admins should touch.

Although this standard is aimed at Linux, it's fairly straighforward and similar methods are employed by Windows Server and MacOSX Server so that is should be possible to create a pluggable support mechanism that is broadly applicable.

the XDG Filesystem Hierarchy defines some environment variables and defaults for the most common types of non code data, as well as offers a system for separating user configuration from global configuration.

I recommend you review the standard, since it's very short, but here's a summary.  The standard defines 4 enviroment variables useful to us:

XDG_DATA_HOME

These is the location of data oriented files that a user running the application should be able to customize (or will be customized during installation or use of the application).  By default these go into "~/.local/share".

XDG_CONFIG_HOME

Similar to XDG_DATA_HOME but specifically for configuration files.  Defaults to "~/.config".

XDG_DATA_DIRS

Takes a string of paths (delimited by ":") where to local for systemwide data.  These could be things like templates or static assets that shouldn't be changed by users and that would be shared by all instances of the application.  The default is: "/usr/local/share/:/usr/share/".

XDG_CONFIG_DIRS

Like XDG_DATE_DIRS but for configuration.  Defaults to "/etc/xdg".

The way I'd see this working is that if the application we being run in development mode, we'd first look for files local to the application file path, and then fall back to looking at the XDG defined directories.  Additionally, we'd probably need some boilerplate install scripts that authors can use to prompt for the desired path information (which rational defaults).  So our application distribution would possible look like:

MyApp-Web
    /t
    /etc
        myapp-web.conf
    /share
        /local
       
    /lib
        /MyApp
            Web.pm
            /Web
                /Model
                /View
                /Controller

And during installation we'd copy "MyApp-Web/share/local" to "$XDG_DATA_HOME/myapp-web" and "MyApp-Web/share/" to "$XDG_DATA_DIRS/myapp-web" (we'd either just copy to the first one in the path or prompt at install time).  Handling configuration would be a bit trickier.  My thougth here is that we'd copy "MyApp-Web/etc/*" to "$XDG_CONFIG_DIRS/myapp-web" but when running the application would like in both XDG_CONFIG_DIRS and XDG_CONFIG_HOME, merging both to allow locally overriding of the configuration.   

Overall I believe this will give us a smoother and more professional installation experience, make it easier to administer Catalyst applications and help start a best practices dialog.

Thoughts, criticism, abuse welcome :)

Last night (NYC time at least) I was able to have a great brainstorming conversation with rafl, confound and others regarding new ways to declare Catalyst Actions.  This conversation was set off by a blog post and my response.

A few things came up in the conversation that warranted me stepping back and pondering the best way to clarify my thoughts and my position.  The first has to do with my feelings about how controllers and actions should require minimum (or ideally no) additional syntax knowledge, above what MooseX::Declare and Moose expects, in order for a developer to be productive.  This is what I meant by 'Controller Baby Talk', and I think it's important since we have a goal of making it easy to get started and perform the common cases.  It's related to the reason why I think query and body parameters should be expressed in the method signature as well. The idea is less method calls, more declaration and more useful defaults. Someone just starting should be able to make decent controllers off the top with a minimum of learning.  They might not make great controllers, but they will make workable controllers.

The second issue has to done with Catalyst Chained Actions which I refer you to the link if you are not familiar.  Chained actions are great in that they help reduce a lot of repeated code, as well as having a very elegant feel to them.  However, in practical life I've not found using them to be very beginner friendly.  I've even seen intelligent, intermediate and above level developers struggle with them.  I hesitate to criticise here, since I'm not contributing code to try and solve these problems (yet) but in general my experience with Chained actions leads me to conclude that they belong in the hands of at least intermediate or higher programmers, people with the experience to realize they need more and the ability to go learn how to do it correctly.  I realize you may feel differently, so please aim your thoughts into comments.  I'm saying this based on my experiences as a team lead building a large Catalyst driven website where I spent too much time struggling to help other developers get their chained actions working correctly, as well as debugging trouble with using $c->uri_for against action chains.

Based on this, I left chained actions out of my discussion.  I do think whatever we create to replace the current method attribute system must address the need to make Chained actions work and work better, I just think it's not a button I want newcomers to the framework to have access to on day one.  That's why I originally left out any discussing of chained actions out of my blog post.

Based on the IRC chat and the feedback I've recieved, I think that you should be able to easily 'scale up' to chained actions as you need them.  In particular, I like the idea, mentioned a few times in last night's chat, that all actions are reall chained actions, just some are very short chains.  Under this thinking, everything is an action chain, but we have some shortcut types to enable some simple cases (such as what the 'Path' attribute is now used for).  These special cases are just shortcuts that behind the curtain uses chaining.  In this case we'd all be using chained actions all the time, but for the simple cases the work would be hidden and the end developer doesn't even know it, until they rise in skill and have that wonderful, "AH HA!" moment of realization!  I can see that could make our underlying code base more clean, as well as remove a lot of special case processing.  As a side effect, it would probably make action chains more robust, and probably solve some my concerns.  That of course would require a major rewrite of the dispatcher code, which is something for another blog.

However, I think we should be careful to move Catalyst controllers too far away from plain old Perl / Moose classes.  Although I don't love the method attribute syntax of the current system, one thing I do love is that Catalyst Controllers are just plain classes with a bit of decoration.  That makes it easy for me to apply all my knowledge regarding designing and developing classes to Catalyst Controllers.  It's something I would not want to see disappear.

Given this idea, I want to offer another thought experiment that would outline the growth from simple case controllers up to chained controllers, trying to follow the intelectual growth of a new developer.  Again, I want to keep the new syntax to a minimum.  Some of the syntax here comes out of the IRC conversation I've mentioned, but no code is written and many more use cases need to be written and expressed before we can clarify the specification.

Let's start with a cleaned up version of the controller I from my last post.  This is a controller that offfer URI endpoints to view a Gallery of People, see details on a particular person (by ID or Email) and create a new person.  Let's assume the Person entity has the following attributes: ID, Name, Birthdate, Email with ID and Email as unique and all fields are required.  Let's also for the moment assume that ID is auto generated by the storage system, such as a database.  I think a webpage like this is a pretty common ask for a developer, and many websites are just more complex versions of this simple case.

    use Catalyst::Declare::Controller;

    class People {
        method gallery($ctx: Int :$page = 1, Int :$rows = 1) {
            my $people_rs = $ctx->
                model('People')->
                search({},{page=>$page, rows=>$rows});
            $ctx->stash(
                people_rs = $people_rs,
            );
        }
        multi method person($ctx: Int $id) {
            if(my $person = $ctx->model('People')->find(id=>$id)) {
                $ctx->stash(person=>$person);
            } else {
                $ctx->forward('not_found', ["Person of id: $id doesn't exist"])
            }
        }
        multi method person($ctx: Email $email) {
            if(my $person = $ctx->model('People')->find(email=>$email)) {
                $ctx->stash(person=>$person);
            } else {
                $ctx->forward('not_found', ["Email: $email doesn't exist"])
            }       
        }
        method create($ctx: Dict[name=>Str, email=>Email, birthdate=>DateTime] $post) {
            my $validated_post = $ctx->model('ValidatePersonCreatePost')->validate($post);
            if($validated_post->is_valid) {
                $ctx->model('People')->create($validated_post);
                $ctx->forward('CreatedNewPerson', ["I created person with $validated_post"]);
            } else {
                $ctx->stash(template=>'people/create_has_errors');
            }
        }
        method prepare_database(HashRef $opts) {
            ## just a plain old method, move along
        }   
    }

I added a bit more of the kind of logic you might expect to see in this kind of controller.  For this case, I assumed we are storing all our data in DBIC, so I used that syntax.  However, I want to point out it's not good practice to put this kind of logic in your controllers!  I'm just doing it this way to make it easier for you to follow along and I will offer one last version at the end which will reflect this best practice.  This will make it easier later on to see the value of chained actions as well.  This controller would map the following URLs:

{hostname}/people/gallery?page={page}&rows={rows}
{hostname}/people/person/{id}
{hostname}/people/person/{email}
{hostname}/people/create

Remember, we suggested in the previous blog that if the last argument in the signature is a reftype, such as something inheriting from ArrayRef, HashRef or even Object, we assume this action requires HTTP Post and attempts to coerce the post entity against the given type constraint.

For the purposes of clarity, I skipped all the MooseX::Types imports, just assume they are there and that useful coercions exists, such as something to parse strings into DateTime objects and something to take a urlencoded post entity and convert it to a HashRef.  For the purposes of making it easier for newcomers, we might even wish to consider automatically importing a bunch of the most common case MooseX::Types and coercions.

One thing that I changed from the last example is I've removed the 'action' keyword, relying exclusively on 'method' which we get from MooseX::Declare.  Basically what I am saying here is that an action is just a method that expects to be part of a context, so just requiring "$ctx:" in the method signature invocant should be enough to tell the underlying system this is an action, not an ordinary method.  I think this could be more intuitive to a newcomer, saying that an action is just a method with a context.  Okay, so that's not the entire story, but again, it's still a pretty big gulp for a new developer.  She can start with that and be very productive with just that amount of knowledge.

I also swapped 'controller' for 'class' which again comes straight out of the MooseX::Declare syntax.  Right now I am not seeing what calling this controller is buying us, except that it a new thing to learn.  Right now it feels gratuitiously different.

So right now you'd have a workable controller and you didn't have to learn anything above and beyond what Moose and MooseX::Declare expect.  I expect you are looking at this and thinking, "Hmm, if we had chained actions, we'd be able to reduce a lot of this repeated model calls."  Also, what happens when you need more complex controllers?  Let's say  the developer gets a request to add the ability to both edit and delete individual Persons within the set of People?  Again, this is a classic argument for Chained actions, since we can have much clearer code.  For the moment though, let's stay with the current knowledge set and see what our developer might write.

    use Catalyst::Declare::Controller;

    class People {
        method gallery($ctx: Int :$page = 1, Int :$rows = 1) {
            my $people_rs = $ctx->
                model('People')->
                search({},{page=>$page, rows=>$rows});
            $ctx->stash(
                people_rs = $people_rs,
            );
        }
        multi method person($ctx: Int $id) {
            if(my $person = $ctx->model('People')->find(id=>$id)) {
                $ctx->stash(person=>$person);
            } else {
                $ctx->forward('not_found', ["Person of id: $id doesn't exist"])
            }
        }
        multi method person($ctx: Email $email) {
            if(my $person = $ctx->model('People')->find(email=>$email)) {
                $ctx->stash(person=>$person);
            } else {
                $ctx->forward('not_found', ["Email: $email doesn't exist"])
            }       
        }
        method create($ctx: Dict[name=>Str, email=>Email, birthdate=>DateTime] $post) {
            my $validated_post = $ctx->model('ValidatePersonCreatePost')->validate($post);
            if($validated_post->is_valid) {
                $ctx->model('People')->create($validated_post);
                $ctx->forward('CreatedNewPerson', ["I created person with $validated_post"]);
            } else {
                $ctx->stash(template=>'people/create_has_errors');
            }
        }
        method edit_person($ctx: Int $id, Dict[name=>Str, email=>Email, birthdate=>DateTime] $post) {
            if(my $person = $ctx->model('People')->find(id=>$id)) {
                my $validated_post = $ctx->model('ValidatePersonEditPost')->validate($post);
                if($validated_post->is_valid) {
                    $ctx->model('People')->update($validated_post);
                    $ctx->forward('UpdatedPerson', ["I updated person $id with $validated_post"]);
                } else {
                    $ctx->stash(template=>'people/create_has_errors');
                }
            } else {
                $ctx->forward('not_found', ["Person $id doesn't exist"]);
            }
        }
        method delete($ctx: Dict[id=>Int]) {
            if(my $person = $ctx->model('People')->find(id=>$id)) {
                $person->delete;
                $ctx->forward('DeletedPerson', ["I deleted person $id"]);
            } else {
                $ctx->forward('not_found', ["Person $id doesn't exist"]);
            }
        }
        method prepare_database(HashRef $opts) {
            ## just a plain old method, move along
        }   
    }

This controller now maps the following URLs:

{hostname}/people/gallery?page={page}&rows={rows}
{hostname}/people/person/{id}
{hostname}/people/person/{email}
{hostname}/people/create   (expects POST entity)
{hostname}/people/edit_person/{id}  (expects POST entity)
{hostname}/people/delete {expects POST entity}

To make this a bit shorter, I said edits and deletes could only occur using a Person ID, even though I allow view access via both the ID and Email.  Now, these are not the most concise and beautiful controllers (there's a bit of repeated code), but they have the advantage of: 1) straightforward for newcomers, and 2) they remain basically plain old Perl / Moose.

At this point, we could offer a refactoring using standard Perl techniques, such as pulling out the repeated code blocks.  This would reflect a developers growing confidence and ability.  I will leave that as an exercise for the reader, since it's offtopic for the current thought experiment.  I only mention it because I want to point out that it would be possible for a developer to gradually increase their ability and get rewarded with better code along a more gradual learning curve.

At this point, now that our developer is starting to become more familiar and confident with the system (given the ease of success so far) she starts exploring the store of community best practices, trying to see what could be done to make these even better and easier.  She comes across chaining, and with some research and help managed to perform the following refactoring (I'm using a possible chained syntax which has not yet been fully research, but hopefully you can get the idea:

    use Catalyst::Declare::Controller;

    class People {

        Person->config(actions => {
            root => {
                name => '';
            },
        });

        method root($ctx:) {
            $ctx->stash(people_rs => $ctx->Model('People'));
        }

        under root($chained_ctx:) {
            my $people_rs = $chained_ctx->stash->{people_rs};
            method gallery($chained_ctx: Int :$page = 1, Int :$rows = 1) {
                   $chained_ctx->stash(
                    people_rs = $people_rs->search({},{page=>$page,rows=>$rows}),
                );
            }
            method create($chained_ctx: Dict[name=>Str, email=>Email, birthdate=>DateTime] $post) {
                my $validated_post = $chained_ctx->model('ValidatePersonCreatePost')->validate($post);
                if($validated_post->is_valid) {
                    $people_rs->create($validated_post);
                    $chained_ctx->forward('CreatedNewPerson', ["I created person with $validated_post"]);
                } else {
                    $chained_ctx->stash(template=>'people/create_has_errors');
                }
            }
            method delete($chained_ctx: Dict[id=>Int]) {
                if(my $person = $result_rs->find(id=>$id)) {
                    $person->delete;
                    $chained_ctx->forward('DeletedPerson', ["I deleted person $id"]);
                } else {
                    $chained_ctx->forward('not_found', ["Person $id doesn't exist"]);
                }
            }
               multi method person($chained_ctx: Int $id) {
                if(my $person = $people_rs->find(id=>$id)) {
                    $chained_ctx->stash(person=>$person);
                } else {
                    $chained_ctx->forward('not_found', ["Person of id: $id doesn't exist"])
                }
            }
            multi method person($chained_ctx: Email $email) {
                if(my $person = $people_rs->find(email=>$email)) {
                    $ctx->stash(person=>$person);
                } else {
                    $ctx->forward('not_found', ["Email: $email doesn't exist"])
                }       
            }
            under person($chained_ctx:) {
                my $person = $chained_ctx->stash->{person};
                method view ($chained_ctx:) {
                    ## just goes to a view template
                }
                method edit($chained_ctx: Int Dict[name=>Str, email=>Email, birthdate=>DateTime] $post) {
                    my $validated_post = $chained_ctx->model('ValidatePersonEditPost')->validate($post);
                    if($validated_post->is_valid) {
                        $person->update($validated_post);
                        $chained_ctx->forward('UpdatedPerson', ["I updated person $id with $validated_post"]);
                    } else {
                        $chained_ctx->stash(template=>'people/create_has_errors');
                    }
                }
            }               
        }
        method prepare_database(HashRef $opts) {
            ## just a plain old method, move along
        }
    }

These map the following URLS:

{hostname}/people/gallery?page={page}&rows={rows}
{hostname}/people/person/{id}/view
{hostname}/people/person/{email}/view
{hostname}/people/person/{id}/edit (expects POST entity)
{hostname}/people/person/{email}/edit (expects POST entity)
{hostname}/people/create   (expects POST entity)
{hostname}/people/edit_person/{id} 
{hostname}/people/delete {expects POST entity}

You probably noticed I changed a lot of the "$ctx:" invocants to "$chained_ctx:"  This is to reflect that the method is part of a chained action context, and different from 'normal' actions.  I think it's a good idea to express this difference clearly, since an action that is part of a Chain does have different characteristics than a stand alone action.  for example, you can't $ctx->forward to it.  I'm not attached to particular invocant name, but I'm just trying to indicate how this could be done without falling back on adding modifiers on the method (such as 'on', 'at' or others).

Overally, this refactor I think is an improvement, since it has less repeated code and the flow across the actions is more clear.  It will make it easier to do even more complicated things later. 

Well, that's mostly it for the thought experiment.  I think I got my basic idea across.

I said earlier in this post that these controller examples have a fatal flaw, which is the amount of logic that rightfully should stay in the model.  My suggestion, if you are using DBIC, is that you treat DBIC methods (search, find, and all that) as sort of private or protected to your result and resultset classes.  In other word, your schema should define an API to expose to the world, and the world should respect that or suffer.  Additionally it's probably suspect to pass a resultset or row object to your Template handler.  This can lead to all sorts of template abuse.  I promised a last refactor, but as I look into the eager eyes of my Akita dog, who is gazing expectly out the window onto a beautiful sunny day, I just know I'll leave that one to another blog :)

Share and Enjoy!

---UPDATES ---

It was pointed out to me that using the method signature invocant in the way I have described doesn't really reflect the true nature of the invocant.  As an alternative we could say that if a method signature has a first, positional parameter which is a Catalyst Context, then the method is an action.  Would look like:

    method myaction(Context $ctx, ....) {}

This would have the intriguing possibility of having subclasses on the Context subtype, allowing multi method polymorphism for the same URL with different information in the context.  This is an alternative to what was described above and may be a better option.

Additionally, with recent builds of MooseX::Declare, methods now support a version of a 'returns' modifier, which is a type constraint on the return value of the method.  Although this itself is not always useful for Catalyst actions, it might be valuable to place a modifier to check the contents of the Stash.  This would add a bit of sanity to the otherwise anarchy of the Catalyst Stash.  Consider:

    method person(Context $ctx, Int $id) stashes(Person :$person ) {
        $ctx->stash(person=>$ctx->model('People')->find($id));
    }

Thoughts welcomed!

The recent release of Perl Catalyst 5.8 has does wonders for our ability to incorporate new developers and new ideas into the Elegant Perl Web Application Framework.  As I have reflected on in other posts, Moose based projects offer a shared context that eases the learning curve barrier for newcomers.  I look forward to the coming explosion of thoughts and ideas!

One idea that keeps coming up is how to replace the existing method attribute system used by Catalyst Controllers to map Actions to URIs into something more flexible and sustainable.  If you don't know what I mean you should read here.  Basically, Catalyst Controller classes use these method attributes as a way to 'tag' actions with the information necessary to tie a given controller + action to a URI endpoint.  This system, which has endured from the first versions of Catalyst (brought over from  Catalyst's inspirational ancestor, Maypole) is starting to show its weaknesses.  Although attributes are cute, this is a system unique to Catalyst controllers, so it doesn't make good use of leveraging code and knowledge from other more widely know skills.  For more about troubles with the current method attribute syntax see Declare Catalyst Actions.

A possibility to fix this problem is to create a sort of mini language crafted specifically for the job.  This would seem to bring Catalyst more in line with how some other popular frameworks are trending, or even exceed them in some ways.  As was recently said to me in IRC, “What I want is the maximum power and flexibility.”  From my perspective, I'd rather make controllers easier for the most common cases, and create a system that works in a usable manner even if all you know is how to build normal methods.  People with a bit more ability and knowledge should be able to influence or 'hint' the system in order to achieve custom effects, but in the best tradition of Perl you should be usefully productive with minimum knowledge.  All the dangerous flexibility is placed one level up the skill scale.  As I once read regarding Perl, when the developer is still a baby, baby talk should be adequate to achieve what a baby needs.  The knowledge needed to do more then can act to better guide them to develop the skill and best practices necessary.

In order to achieve this “Baby Talk” controller,  we need a deeper and more fundamental rethinking of the role of Controllers and how they function, and particularly how creating Controllers fit into the development process.  Simply adding a sleek mini language is not going to address what I think of as these main problems.  So let's, as a thought experiment, consider how we could create controllers with the minimum expected knowledge.  Then, as a second step, let's try to talk things further and see if we can create a controller system that is equally useful for web, command line, GUI or other types of interfaces.

First, here's a 'classic' Catalyst Controller:

    package MyApp::Controller::People;
    use base 'Catalyst::Controller';

    ## URL = $base/people/gallery?page={page}&rows={rows}
    sub gallery :Path('gallery') {
        my ($self, $c) = @_;
        my ($page, $rows) = @{$c->request->query_parameters}{qw/page rows/};
    }

    ## URL = $base/people/person/{id}
    sub person :Path('person') Args(1) {
        my ($self, $c, $id) = @_;
    }

    ## URL = $base/people/create + POST entity
    sub create :Path('create') {
        my ($self, $c) = @_;
        my ($name, $age) = @{$c->request->body_parameters}{qw/name age/};
    }

    ## Plain method, not an action
    sub prepare_database {
        my($self, %args) = @_;
    }

Assume the controllers actually do something useful.   The idea here is to show the work involved in mapping a URL and getting some common stuff, like body or post parameters and path arguments.  This gets the job done, although the developer is left with a lot of validation work.  Notice how there's different ways to get the path parameters from the query and post parameters?  In this case the developer has to do a lot of work just to get URI information, and we still have something that's very loosely defined, with lots of room for errors.

Okay, so let's see what we can do if we start with something based on MooseX::Declare instead.  Yes, I know I am presupposing knowledge of this module, but: 1) Catalyst is Moose based now and we can assume the developer knows 'baby talk' Moose, 2) knowledge of MooseX::Declare is useful for more than just controllers and 3) It looks familiar enough to other programming systems that the learning curve should be low.  However, if you don't know MooseX::Declare, you should go take a quick look now.

Here's the Controller.  It's not all “Plain old Perl/Moose” yet, but much closer:

use Catalyst::Declare::Controller;  ## Doesn't exist yet :)

class People {
    action gallery(Int :$page, Int :$rows) {}    
    multi action person(Int $id) {}
    multi action person(Email $email) {}
    action create(Dict[name=>Str,age=Int]) {}
    method prepare_database() {  #just a plain old method, move along}
}

The idea here is that MooseX::Declare, along with MooseX::Method::Signatures, already offers us a pretty powerful system for expressing arguments, and it's highly introspectable.  Why make yet another new signature system, one that is only useful for Controllers, and makes a controller feel like something other than a friendly Perl class?

Now, I know the above example need a bit of explaining, and of course there's gaps, as you would expect at this point.  So, going though each action:

    Gallery:  This has a signature that wants (page=>..., rows=>...) as named arguments and additional expects both those to be an Int (see Moose::Utils::TypeConstraints if you need a primer on Moose TypeConstraints).  Since the Int is a standard Moose type constraint, you can leverage the entire type constrain tool chain, including MooseX::Types and coercions, which gives you a lot of possibilities as well as greatly assisting you in your first level validation.  The URI generator would map the path parts are $base, 'people' and 'gallery' as is already does, but we'd add in that trailing Hash arguments in the signature become mapped to query parameters.  So this makes a URL like “$base/people/gallery?page={page}&rows={rows}”.  This is the default, and is probably very close to what you are looking for.  I think it works intuitively, and has the bonus of helping people see query parameters as something more than a dumping ground.  

    Person:  I tossed in the 'multi' keyword from MooseX::MultiMethods but I think you get the idea.  Again, here's a nice bonus to using Controllers that are just Moose classes , we can take advantage of this great module, and whatever else the future brings along.  In this case both methods want a single positioned parameter, which the URL mapper converts into a path part.

    Create:  This one is the most challenging, but my idea here is that if the final argument in a signature is a collection like type, that means it's a POST or PUT listener, and expects an entity body.  We need a bit more work here, but I think it's very doable.  Imagine how useful this would be for dealing with uploaded entities as well!

Well, how might you perform the hinting I mentioned.  Right now I think this could fit into either the config or possibly as a meta description;

    class Person {
        meta_description(
            actions=>{
                person->{
                    ## would change url to: $base/people/{id}
                    uri => '{base}/people/{id}',
                },
            }
        );

        ## all the rest...
    }

Again, this is still a mock up, but the idea I think is workable, and feels clean to me.  

Another intriguing possibility is that once you cleanly separate your Web decorated controllers from the core logic being expressed, it easy to imagine how you could use the same inference to build command line applications.  For example, the same system above might reflect to a command line tool as:

    myapp.pl people.gallery --page=1 –rows=10

My thinking here is that you'd create a base 'MyApp::People' and then depending on what engines you set up in the configuration, it would generate a 'MyApp::Web::Controller::People' or a 'MyApp::App::Controller::People', or even 'MyApp::SOAP::People', and so on.

Final Thoughts.
 
I'm opposed to creating a new solution for something that already exists in Perl or that exists in a sufficiently large community (such as Moose).  We have a big enough learning curve with Moose and related, and I think the more we add in, the worse off we are.  My feeling is that controllers should be the minimal decoration on top of plain old Perl necessary to map URIs to the work required.  This would make is possible to leverage the entire existing Moose tool chain as well as the collected wisdom of Moose development best practices.  Additionally, since Moose has such a rich set of introspection ability, this should make it possible to make a reasonable inference as to how a given controller maps to a URI even without any additional meta information.

Overall, this has been a thought experiment.  There's no code behind it yet.  My goal here is to see if I can get you to think differently about how your projects are organized and see if we can come up with something that really is ahead of the the current web development architectures.  I really believe that although Model – View – Controller is a solid paradigm for web development, our implementations of it have so far been too tyrannical and have gone down paths that cause confusion as well as make it hard for Perl programmers to use the knowledge they already have.  A system as described above would avoid those issues, yet increase options and flexibility, while making it easier to just get started.

All the bits are in place and I could really use some help getting notice for the Catalyst 5.8 Press Release so please submit it widely and add it to your http://delicious.com/ bookmarks or whatever other social bookmarking you are using.  Let's try to get some press for all our hard work.

So far It's gone to:

use perl (pending)
slashdot (pending)
digg (http://digg.com/programming/Perl_Catalyst_Web_Application_Framework_Version_5_8_Release)
reddit (http://www.reddit.com/r/programming/comments/8h511/perl_catalyst_web_application_framework_version/)
ycombinator (http://news.ycombinator.com/item?id=589095)

I'll update this as I get more accepts.

The following is an edited and updated version of a Moose advocacy bit I wrote on the Google AppEngine for Perl project mailing list quite a while ago.  I decided to republish it here since with the recent release of Catalyst 5.8, which uses Moose as a core technological foundation, I felt it would be a good idea to reflect on why I think this is a great thing.  There are quite a few tutorials and code overviews for why Moose is a superior object system (links at the end of the article) but I don't see many postings that are simple reflections on Moose and why I, as a developer, am happy it belongs to Perl.

I also think the Google AppEngine for Perl project is not getting nearly enough love and I'm hoping to draw a bit of attention to it as well.

You can read the original here, but that version has an embarrassing number of spelling and grammar errors :)

Without Moose, I'd probably have lost the faith and left Perl for another language.  This is not a lightly written statement, since I've been using Perl as a primary server side development language since 1996.  When I first started programming I gravitated toward Perl for two reasons:  First, it let me do what I needed to do without getting in the way or forcing me to jump through a lot of hoops.  Second, it introduced me to what is now broadly thought of as the Open Source community, an idea of how software development and knowledge sharing could be that excited my idealistic nature.  I have to imagine there are quite a few programmers in Perl with a similar story.  A lot of us started using Perl because the module CGI made it trivial to create server side web forms, while DBI created a straightforward means to shuffle information between the web and back end databases.  Perl did what I needed it to do.

However, over the time, my love affair with Perl diminished.  Perl's default Object Oriented system is very minimalistic and I found myself envious of languages that allowed --and even encouraged-- a best practices approach in modeling your application.  For the first time I started to feel like Perl was getting in the way of me just doing what I needed to do, mostly because I was spending a lot of time jumping through hoops and not expressing my problem domain.  Even worse, this lack in Perl caused many large projects to create there own ways to close the gaps left open by existing methodologies.  How many different and incompatible component systems have been written for Perl?  How many systems for making it easier to do object oriented programming?  And each time you go to load a new module from CPAN, you end up also loading a bunch of dependencies for yet another half baked OO framework that is only being used on a handful of projects.  This reality makes it much harder for newcomers to usefully contribute code, since each time you have to learn a new underlying framework and all its idiosyncrasies.  That's why most important Perl CPAN modules have at most a small handful of contributors.  Worse, if fed the common belief that Perl is spaghetti code not suitable for larger applications.  Now, I do think having competing ideas is good, and that value is one of the things that makes Perl strong, however for basic, core programming tasks the lack of consistency really hurts us.  Where would Perl be if CGI had not been so complete, or if DBI did not become the dominate method of accessing databases?

Moose has reinvigorated my love for the Perl language, it's community and programming in general.  It cleanly solves the problem of Perl's minimal OO features.  I write less boilerplate and spend more time expressing my problem domain.  It provides a wealth of expression around which a rich economy of best practices and shared solutions can grow.  Additionally, since it's written for programmers by programmers, I find that it fits my brain better than any other ivy tower OO solution.  It has a large and growing community of active developers, more than any other competing system OO framework for Perl AFAIK.  Lastly, some of the most important Perl projects of our day, such as Catalyst, DBIx::Class, etc are either using Moose or moving toward Moose.  For me, Moose makes OO Perl just as expressive and fun as I found Perl to be when I first started writing website applications all those years ago.

Is it perfect?  No, certainly there is a lot of room for Moose to improve.  However, all the right decisions to let those improvements happen have been made.  The developer community is very serious about feature enhancements and about growing adoption of the framework as well as growing the number of contributors.  I am a case in point, since although I've been using Perl for more than 10 years, Moose is one of a small number of projects that I've managed to become involved at the contributor level.  Moose not working the way you need it to?  That's fine, come to IRC and make your case.  However, I would encourage to do your homework first, since the Moose channels have some of the brightest minds I have ever had the chance to see in action.

For me, properly written Perl starts with Moose.  It should serve as the basis for any serious project you may be about to undertake.

• For more about Perl.
• For more about Moose.
• The Official Moose Learning Manual
  
• For more about the Google AppEngine for Perl project.

Here's an article you might have missed.  Nice summary of the main features of the 5.8 version of Catalyst.