Thursday 30 April 2015

My March & April CPAN PR Challenge dists

As I got ready to blog about my PR Challenge assigned dist for April, I realized I had forgotten to blog about my last month's assigned dist, so I have decided to write a joined march/april report :)

March distribution: Web::Library::UnderscoreJS


This neat module by Marcel Grünauer is a wrapper for the Underscore JavaScript library, which itself provides over a 100 functional helpers and goodies for your frontend programming like function binding, JS templating, quick indexes, and much more.

The wrapper comes from Web::Library, a general manager for client-side dependencies. Say you have a web application. You spawn the Web::Library singleton and mount the libraries you want to use (Underscore in our case):

    my $library_manager = Web::Library->instance;
    $library_manager->mount_library({ name => 'UnderscoreJS' });


Then you set the include_paths() from the instance as directories to search for static files. In Catalyst, this can be achieved with Catalyst::Plugin::Static::Simple, like so:

    __PACKAGE__->config(
        'Plugin::Static::Simple' => {
            include_path => [ $library_manager->include_paths ] },
        ...
    );


Then pass the instance to your templates and call the appropriate methods to print the required CSS/JS lines:

    <head>
        ...
        [% web_library.css_link_tags_for('UnderscoreJS') %]
   
</head>
    <body>
        ...
        [% web_library.script_tags_for('UnderscoreJS') %]
   
</body>

My pull requests for this distribution involved bumping up the copyright year, some minor pod formatting issues, updating the UndescoreJS library versions (up to 1.8.2), and adding a Travis-CI code badge. I also investigated some CPAN Testers failure reports for this distribution on MS Windows, but they turned out to come from Web::Library itself. Nevertheless, I also send a PR with a fix and another one that reduced the minimum perl version to 5.6, which was asked in an open issue.


April distribution: Mojolicious::Plugin::ServerStatus


This is a plugin for Mojolicious that shows server status information, similar to Apache's mod_status. It is based on the general Plack::Middleware::ServerStatus::Lite middleware and works on multiprocess servers like morbo and hypnotoad. It's not able to monitor keepalive sessions and network I/O wait, but it still gives some nice output in different formats.

All you have to do is load the plugin and tell it which route path to render. Optionally, for security, you can set which IPs to allow:

    plugin 'ServerStatus' => {
        path => '/server-status',
        allow => [ '127.0.0.1', '192.168.0.0/16' ],
    };


That's it! After the server is up, all you have to do is query the proper path:

    % curl http://server:port/server-status
    Uptime: 1234567789
    Total Accesses: 123
    BusyWorkers: 2
    IdleWorkers: 3
    --
    pid status remote_addr host method uri protocol ss
    20060 A 127.0.0.1 localhost:10001 GET / HTTP/1.1 1
    20061 .
    20062 A 127.0.0.1 localhost:10001 GET /server-status HTTP/1.1 0
    20063 .
    20064 .


You can also ask for JSON output format:

    % curl http://server:port/server-status?json
    {"Uptime":"1332476669","BusyWorkers":"2",
     "stats":[
       {"protocol":null,"remote_addr":null,"pid":"78639",
        "status":".","method":null,"uri":null,"host":null,"ss":null},
       {"protocol":"HTTP/1.1","remote_addr":"127.0.0.1","pid":"78640",
        "status":"A","method":"GET","uri":"/","host":"localhost:10226","ss":0},
       ...
    ],"IdleWorkers":"3"}


Unfortunately, this distribution didn't have any contents in the main github repository, so instead of making several branches off of master, I used my own master branch and made one commit per change in a single pull request. The first commit of course was just a sync between the repository and the version available on CPAN. Other commits included updating the README, setting up the proper dependencies (which should fix current failure reports from CPAN Testers), adding a Changes file which was missing from the dist, and some minor pod fixes.

That's it! Hopefully those commits and pull requests will be useful to the authors (and users too). And now I can't wait to see what Neil Bowers have picked for me next month :)


Saturday 14 March 2015

How to add online code badges to your Perl projects


If you hang around Github you might have seen a lot of projects adding extra information to their README files in the form of badges, like so:

screenshot of badges shown on Clone's README file













Not only do they make your project's Github page look cooler, they also help developers, from curious potential users to regular contributors, easily spot how active and healthy is the code.

Wanna join in on the fun? There are several free code badges out there for you to use, and these are the ones (that I know of) that work for your awesome Perl code! \o/

Travis CI


Travis CI is a continuous integration service used to build and test projects hosted on Github. They'll let you know whether your code builds and tests pass in many different versions of perl (but, unlike CPAN Testers, they'll do it in only one platform). Using their service is super simple:

1. Go to the Travis CI homepage and sign in with your Github account. It's the big link in the top right corner of the page, you can't miss it :)

2. If this is your first rodeo, you'll see Travis CI asks for some permissions on Github. You must allow it to proceed. They have outlined the whys and hows if you're feeling cautious.

3. Go to your Travis CI profile page and wait for all your repositories to sync. Once they're all listed in there, it should look something like this (with, of course, your own name, groups and projects):

 4. See that "on-off" switch next to each of your repositories? Just flick the ones you want to have a Travis CI report. We're almost done!

5. Add a file named .travis.yml to the root of your project repository (note the "." in front of the filename!). It should contain valid YAML with the main language and the different versions of Perl 5 you want to test against. For example, if you want to test against 5.14 and 5.20, your ".travis.yml" should look like this:

language: perl
perl:
  - "5.14"
  - "5.20"

If you need any extra information or customization of your perl builds, just read Travis CI's Perl specific documentation. Note that if something goes wrong (e.g. you make a typo in the filename or its contents), Travis will assume you're testing against the current stable version of Ruby, which is definitely not what you want :)

A few moments after you commit it to your repository and push it to Github, go back to your Travis CI profile page and you should be able to see the build results! If you skipped a step and added the ".travis.yml" file to your repo before enabling the project in the Travis CI page, just push any commit to trigger it.

6. Badge time! Make sure your README file is in Markdown - just rename it to README.md and adjust it as you see fit (it's also a great way to make your project stand out while still making it readable as plain text). Then go to your Travis CI profile and click on the project you want to add the badge. You should see the badge right next to the project's name. Click on it and it's going to let you copy the badge for your project in several formats:



Choose "Markdown" and paste it on your README.md file. The one from Clone, as you can see from the picture above, looks like this:

[![Build Status](https://travis-ci.org/garu/Clone.svg?branch=master)](https://travis-ci.org/garu/Clone)

After pushing your changes, go to your project's Github page and see the badge in all its glory!

Coveralls


Now that we have Travis CI doing some tests for us, how about we get some test coverage reports also? That's where coveralls.io comes in!

1 & 2. Just like we did with Travis CI, go to the Coveralls home page and sign in with Github, granting them access to your repositories.

3. The Coveralls repository page will list all projects being analyzed for coverage, and now is probably the time to click the "add repos" button on that page and add the ones you want coverage analysis of.

4. Adjust your .travis.yml file to add code coverage. Coveralls is engine agnostic so we'll use the not nearly enough praised Devel::Cover suite to do the hard work, together with the Devel::Cover::Report::Coveralls interface. Basically, Travis CI gives us a lot of granularity while building and testing our projects, so we tweak it to include coverage commands in the appropriate build steps.

There are a few ways to go about it: in the manual way we export the environment variables we need and install the coverage dependencies. For instance, we could update our file above to look something like this:

language: perl
perl:
  - "5.14"
  - "5.20"

install:
     - export RELEASE_TESTING=1 AUTOMATED_TESTING=1 AUTHOR_TESTING=1 HARNESS_OPTIONS=c HARNESS_TIMER=1
     - cpanm --quiet --notest Devel::Cover::Report::Coveralls
     - cpanm --quiet --notest --installdeps .

script:
     - PERL5OPT=-MDevel::Cover=-coverage,statement,branch,condition,path,subroutine prove -lrsv t
     - cover

after_success:
    - cover -report coveralls


in the automatic way, we delegate most of the work to some fine-tuned helper scripts created just for Perl projects on Travis and Coveralls. The same original .travis.yml file would, in this case, look like this:

language: perl
perl:
  - "5.14"
  - "5.20"
matrix:
  include:
    - perl: 5.20
      env: COVERAGE=1
before_install:
  - eval $(curl https://travis-perl.github.io/init) --auto


To me this one looks much easier to read, as most of the complexity is enclosed in the "init" script of the well maintained travis-perl project. I highly recommend it. In fact, after you get it right for your code, I suggest you go read their helper intro and play with some settings, you won't regret it. Also, check out Moo's .travis.yml file for a glimpse of all the cool stuff you can get, from testing against blead perl to reporting on a specific irc channel!

5. Now that your commits are tested on Travis CI and reported by Coveralls (mind you, it might take several minutes for a build coverage to show up, and they only start after you push to Github *after* having activated Coveralls and pushed your .travis.yml file), let's put a badge on your README!

On the project listings on Coveralls, click on the project to go to its page, and you should see a badge right at the top, with a "BADGE URLS" link next to it. Click on that link to copy your badge in several different formats, like so:


Put the markdown code on your README.md file next to the Travis CI badge (or wherever you see fit) and that's it!

Version badge


This one really comes in handy for Github, as it lets your users know exactly what's the current stable version on CPAN and how to get it, and it's also by far the easiest to set up: just go to the Version Badge link for Perl projects and type in the name of your target dist (note that it's The-Distribution-Name, not The::MainModule::Name). Once you click on "Find", it's going to give you the badge in several different formats ready for use:


Showing your badges on MetaCPAN


Now that your README.md is all shiny and colorful with badges, your project's Github page is starting to look even cooler than the MetaCPAN page, and that's just not cool. MetaCPAN's web interface is awesome and it will look even better when you put your brand new badges in it, so let's get on with that.

Perl's Pod format allows us to to specify sections that will only show when a given format is supported. For instance, if you write this in your pod:

=for html
<img src="http://example.com/path/to/some/image.png" />

Then this block will render the image when the renderer supports html (like the MetaCPAN web interface), and not do anything on renderers that don't support it (like your command line "perldoc"). Pretty cool, right? Graphic modules and tutorial-like documentation have been taking advantage of this feature for years, and we should too!

So go to the links above and find the HTML version of the badges, then put them in a "=for html" paragraph at the top of your Pod. For the Clone module they look like this:

=head1 NAME

Clone - recursively copy Perl datatypes

=for html
<a href="https://travis-ci.org/garu/Clone"><img alt="Build Status" src="https://travis-ci.org/garu/Clone.png?branch=master" /></a>
<a href="https://coveralls.io/r/garu/Clone?branch=master"><img alt="Coverage Status" src="https://coveralls.io/repos/garu/Clone/badge.png?branch=master" /></a>
<a href="https://metacpan.org/pod/Clone"><img alt="CPAN version" src="https://badge.fury.io/pl/Clone.svg" /></a>

=head1 SYNOPSIS 
 
   ... 

As soon as you push the new version to CPAN, you'll see the badges right there on MetaCPAN!
















If you are too lazy to go back and copy the HTML version from all those sites, just run the following one-liner on your README.md to print the html version of the markdown links in there:

perl -nE 'BEGIN{ say q(=for html) } say qq(<a href="https://www.blogger.com/$3"><img alt="$1" src="$2" /></a>) if /\[!\[([^\]]+)\]\(([^\)]+)\)\]\(([^\)]+)\)/' README.md

Then copy the block to your pod and make a new CPAN release :)

That's all there is to it. Have fun with your badges! \o/

(oh, and let me know if you find any other Perl-related badges you know of)

Saturday 28 February 2015

My February CPAN PR Challenge: Template::Plugin::Autoformat

For this month's CPAN Pull Request Challenge I was assigned with  Template::Plugin::Autoformat, a module that lets you easily format text and numbers in your Template Toolkit templates, using Damian Conway's excellent Text::Autoformat. If you ever needed to adjust text right/left/center justification, alignment, capitalization, bullets, indenting, without being able to resort to CSS - for example, if your templates are not for HTML or if said text is inside a <pre> tag - this module could make your life much easier!

A different challenge


It was an unusually busy month for me and I didn't get a chance to tackle it until yesterday - I even got Neil's "One week left!" email reminder, which was nice. Even so, I figured it would be ok, because the original PR Challenge email mentioned this module had several CPAN Testers FAIL reports, CPANTS issues, and hadn't seen an update in several years.

Except now when I finally checked it out and it saw it was last released a month ago, had zero failures on CPAN Testers, no open issue on either RT or Github, no CPANTS issues, pristine documentation, even complete META resource information! I thought I had been assigned to a dist needing help, but instead what I was looking at was a stable and well-maintained one.

As it turns out, Template::Plugin::Autoformat hadn't seen a single update between 2008 and 2014, when it was adopted by Peter Karman. Peter started making several developer releases until he was satisfied with the results, and released a stable version in January 2015 (which is probably why it was still in the CPAN PR Challenge's list, created prior to said release). His new version fixed all open issues, had a great test coverage and felt like one of those modules that do one thing and do it very well.

And now I had 1 day to send a nice pull request to a module that looked like it needed no pull requests, or I'd lose the challenge :X

Starting small


Okay, instead of being caught in analysis paralysis, I cloned the repo and looked for low hanging fruits. Turns out there were some!

  1. The copyright year was still 2014. This is the first place I look because it's also the first place I overlook in my own projects :) It's such a simple patch that it almost feels like cheating, but you have to start somewhere, right? Done.
  2.  The README was not in Markdown. I love markdown READMEs, because they make the Github project page look *so* much nicer without compromising reading it from the terminal. Even better, "README.md" is fully supported by PAUSE & CPAN \o/. As I was making the conversion, I noticed the README's contents were just a copy of the pod, so I tweaked it a bit to include installation instructions and just a teaser pointing to the full docs, online and via perldoc. This is a good thing for the developer too, as there's less duplicate content to worry about. Done.
  3. The distribution did not declare a minimum perl version. CPANTS Kwalitee is a terrific free service for the Perl community, letting users and authors know whether a given module passes or fails several quality assurance metrics. While, as I mentioned before, Template::Plugin::Autoformat passed all core CPANTS metrics, this extra metric was not being met. In fact, it was the only extra metric not being met. Thankfully, the excellent perlver tool makes it very easy to find the minimum perl version for your module or app. It reported 5.6.0 as being the minimum version so, after a very simple addition to the Makefile.PL, I had my third pull request of the night.
  4. The Changes file was not fully compliant with the CPAN::Changes spec. This was also an easy one to fix, since the only standing issue was formatting the release dates to something CPAN::Changes would understand. Next!
  5. Test coverage was almost 100%, but not exactly 100%. This is another great way to help other projects: check the code coverage and see if you can improve it in any way. In this case, after running the great cover tool, I found out it had 100% statement coverage, but 50% pod coverage and 91.6% branch coverage. The pod coverage was actually a mistake - there was a private function being counted as public. Adding the missing branch test was also pretty straightforward. After the patch, Template::Plugin::Autoformat got 100% coverage in everything - which is pretty cool!
  6. The "NAME" key in Makefile.PL had the distribution name, not the main package's name. Now, the builder is clever enough to do the right thing, but nevertheless it was triggering a warning every time I ran "make" - which was quite a bit while I played with test coverage. Easy fix again, just s/Template-Plugin-Autoformat/Template::Plugin::Autoformat/ and I was done for the night.
So after a couple of hours having fun with Template::Plugin::Autoformat, I had 6 PRs to show for on the PR Challenge. Woot! Best of all, just a few hours later Peter Karman merged all my PRs and made a new release \o/

Thursday 29 January 2015

My CPAN Pull Request Challenge for January: Fuse


So! Back in December I subscribed to Neil Bowers' CPAN Pull Request Challenge. In a nutshell, throughout 2015 every participant will get an email in the first days of each month with a CPAN module that needs a little bit of love, and we have until the end of the month to submit at least one pull request for that module's Github repository.

The module I got was "Fuse".

What is Fuse?


I must admit to my ignorance as I have never heard of Fuse before. So the first thing I did was jump to Fuse's page on MetaCPAN and have a look. Fuse is a Perl interface to the FUSE library (Filesystem in USErspace), and lets you implement filesystems in Perl. Wow! That's pretty cool, right?!

Where to start


If Neil was pointing me to Fuse, it means there's probably work for me to do. So what I did was read the documentation, CPAN Testers reports, open issues and the Github page. Then I downloaded the source code and grep'd for tags like "FIXME" and "TODO". In the end it was a very well-thought distribution and I think the authors did - and are still doing - a pretty nice job. Still, this is what I thought I could help with:
  • The (external) FUSE kernel/lib doesn't seem to be linked anywhere in the main doc, just on the README;
  • The SYNOPSIS and DESCRIPTION could be a bit more verbose - maybe I'm just not used to the FUSE library, but I felt a lot was being taken for granted;
  • The README (displayed on github) doesn't point to a lot of useful places in the Perl-sphere (other than the examples and how to install);
  • There is no license information on the main pod - though it's there on the META resources;
  • The latest version is not indexed for some reason;
  • There are a lot of FAIL and UNKNOWN reports on CPAN Testers.

First things first


There were several small things for me to attempt here, so rather than making a huge bundled Pull Request for Fuse, I chose to compartmentalize them into smaller, separate, PRs. This way the authors'd probably be more comfortable reviewing my changes and I wouldn't depend on all of them being accepted. This is super easy with git and github: just create a branch from master and make the pull request from your specific branch to the author's master. This will let you easily go back to their master (instead of to the one you might have changed) and work on something else independently.

1. Doc fixes

Since I was really short of free time this month, I decided to start with the lowest hanging fruit. So I created a "garu/doc_patches" branch and tried my best to improve on the existing documentation. I expanded the SYNOPSIS, put external links whenever I missed them, and did some minor tweaks here and there with the pod formatting to improve readability. I also moved the README to markdown so it looks awesome on Github while still being nice to read on the console. When I was satisfied, I sent out the pull request.

This was great and really helped me understand what Fuse was all about and how to use it. I recommend anyone trying to learn and contribute to a project to first read the docs and try to improve them whenever you find something you don't quite understand. Most of the time the ones writing the documentation are too familiar with the code and API so they can take a lot of it for granted.

2. WHY U NO INDEX?

There was an open ticket saying 0.16.1 was not indexed. Upon further investigation, I saw versions 0.16_01 and 0.16.1 were in fact indexed, but available as if they were older releases. Why weren't they showing up as the latest? I went to #metacpan on irc.perl.org and asked around.

As haarg++ pointed out, Fuse inadvertently mixed single dotted and double dotted version formats, releasing '0.16' (instead of '0.16.0') and '0.16.1'. When the CPAN indexer compares version numbers, 0.16.1 becomes 0.016001 and plain 0.16 becomes 0.160000, and as such, "0.16" is indeed greater than "0.16.1".

While I believe version numbers should be boring, I understand why some people go for the major.minor.revision semantic versioning format. So I went ahead and filed a pull request bumping up the version to 0.160.2, and another one that bumps it to 0.17. Both numbers are greater than 0.16 so they'll definitely show as latest, and the authors can just pick whichever format they prefer and discard the other PR.

3. Making tests pass

Finally, I noticed that all the UNKNOWN tags on CPAN Testers were there because FUSE was not installed. Implementing an Alien::FUSE module seems like a good idea, but it is a bit out of scope for the PR Challenge and the limited free time I had. So I went on and installed FUSE for OS X using homebrew. For my surprise, the Makefile.PL was still not finding it:

    $ perl Makefile.PL
    Cannot build for platform: darwin
    Please install OSXFUSE from http://osxfuse.github.com/

I checked Makefile.PL's source and found out what was wrong: it's trying to find libfuse using the "pkg-config" tool. Although "pkg-tool" is rather ubiquitous in major Linux systems, my OS X did not have it. So a made another PR that checks for the existence of either 'pkg-config' or 'ppkg-config' (a Pure-Perl alternative to 'pkg-config' available with the excellent PkgConfig CPAN module) early on the Makefile.PL, dying with an explanatory message if none are available. There's even an opportunity here to simply depend on PkgConfig and use the module version of the commands instead of relying on external "(p)pkg-config" tools, but I thought I rather wait a bit and see how well received this change is first.

Now the module compiles on OSX, but I still can't make the tests pass. I dug a bit and found some commands like read_buf and write_buf are required for testing, but are only available in libfuse versions 2.9.0 and above. However, osxfuse seems to use a different version equivalence and the latest version (released just a month ago) is still 2.7.4. I couldn't find an osxfuse <=> libfuse version equivalence, and because of this, the Fuse module doesn't create the proper bindings for such functions and the tests fail. There might be other issues still, but I stopped there, filing a ticket and hoping someone else will pick it up where I left off.

Wrapping up


I had a lot of fun finding out about Fuse and playing with it. By the end of January I was able to file 4 pull requests and 1 RT ticket to the main repo. I'm sad I wasn't able to get the tests to pass on OS X, but hopefully someone else will read this and be inspired :)

I'd like to thank Neil for this awesome PR Challenge idea, Graham "plicease" Ollis for the great PkgConfig module and for quickly replying to and merging the small changes I proposed that allowed me to use it on Fuse, and Graham "haarg" Knop and everybody from #metacpan for their amazing support.

Also, of course, a huge thank you to Dobrica Pavlinušić, Derrik Pates, Mark Glines and the entire Fuse team for such a great module, and for allowing me the opportunity to be a part of it.

Can't wait to see which module I'll get next month! :)