2011-01-08

Making a Minting Profile as a CPANized Dist.

... or one more reason why Dist::Zilla is awesome.

Minting With Dzil

Minting is a term we use for "Creating a new distribution from a template of sorts". Since Dist::Zilla 2.101230 ( 2010-05-03 23:42:27 America/New_York ), there has been a command for Dist::Zilla to facilitate setting up a new distribution.

$ dzil new Acme-An-Example
# [DZ] making target dir /tmp/Acme-An-Example
# [DZ] writing files to /tmp/Acme-An-Example
# [DZ] dist minted in ./Acme-An-Example
$ find Acme-An-Example/
# Acme-An-Example/
# Acme-An-Example/dist.ini
# Acme-An-Example/lib
# Acme-An-Example/lib/Acme
# Acme-An-Example/lib/Acme/An
# Acme-An-Example/lib/Acme/An/Example.pm
$ cat Acme-An-Example/dist.ini
# name    = Acme-An-Example
# author  = Kent Fredric 
# license = Perl_5
# copyright_holder = Kent Fredric 
# copyright_year   = 2011
# 
# version = 0.001
# 
# [@Basic]
$ cat Acme-An-Example/lib/Acme/An/Example.pm
# use strict;
# use warnings;
# package Acme::An::Example;
# 
# 1;

And while this is a good starting point, once you've gotten into Dist::Zilla, you'll probably find it somewhat lacking and find yourself doing things over and over again.

For this, we have "Minting Profiles".

Basic Minting Profiles

.

Minting Profiles have been in Dist::Zilla since 4.101780 ( 2010-06-27 14:30:55 America/New_York ).

The first thing you'll tend to do, is make a minting profile in ~/.dzil

dzil.org's minting-profile tutorial adequately covers the innards of how this works, but summarised, its lots like dist.ini. You have a profile.ini in a directory in ~/.dzil/profiles/$profilename/ and it will control how your dist is created, and you can then throw together new dists as simply as:

$ dzil new -p $profilename Acme-An-Example

In the absence of -p $profilename, if there is a ~/.dzil/profiles/default/, that will be used instead.

This is pretty convenient.

Me, I had a very simple default profile for a while that did most of what I needed to do:

; ~/.dzil/profiles/default/profile.ini
[Author::KENTNL::DistINI]

That behaves more-or-less identical to how normal dzil new operates, but it generates a custom dist.ini from my custom Author::KENTNL::DistINI plugin. This is based on DistINI just tuned for how I like it. Its more or less a template ;).

; Generated by Dist::Zilla::Plugin::Author::KENTNL::DistINI version 0.01023312 at Fri Jan  7 20:05:56 2011
name             = Acme-An-Example
author           = Kent Fredric 
license          = Perl_5
copyright_holder = Kent Fredric 

; Uncomment this to bootstrap via self 
; [Bootstrap::lib]

[@KENTNL]
version_major     = 0
version_minor     = 1
; the following data denotes when this minor was minted
version_rel_year  = 2011
version_rel_month = 1
version_rel_day   = 7
version_rel_hour  = 20
version_rel_time_zone = Pacific/Auckland
twitter_hash_tags = #perl #cpan

[Prereqs]

CPAN Centralization

However, many people like to use CPAN as their file storage mechanism, and centralise all their perly bits they might need in different locations so they can just get their current favourite configuration with a handy :

cpanm --interactive --verbose Dist::Zilla::MintingProfile::Author::KENTNL 

A Word on Name-spaces

Before we go further, I think it important to plead my rationale behind the "Author::" prefix I'm using now, as I feel its an important concern.

As we have seen with Dist::Zilla there have been a slew of PluginBundles with CPANID's in their name, to the point that there is a copious amount of name-space pollution in the PluginBundle name-space, and more Author bundles than task-bundles, which was really what the name-space was designed for, and I'll freely admit, I am guilty of this crime as well, but I'm petitioning you to help reduce this annoyance in future modules.

From a CPAN testers perspective, the annoyance of lots of CPANID-dists is similar to the annoyance of the whole DPCHRIST:: subspace, and that if this pattern continues, it will mean for the testers who do not wish to test everyones personal modules, that they will have to work hard to avoid this. If DPCHRIST:: had used something like Author::DPCHRIST:: instead, I doubt so many people would be horrified by it, because you can just have a policy/rule that excludes ^Author::, and everyone else who goes that way can be quietly ignored.

Then we could probably rationally add that same restriction to the irc announce bots, the "recent modules" list and soforth, and possibly even apply special indexing restrictions or something so people wouldn't even have to know those modules exist on cpan!

So, for the sake of cleanliness, semantics, and general global sanity, I ask you to join me with my Author:: naming policy to voluntarily segregate modules that are most likely of only personal use from those that have more general application.

Dist::Zilla::Plugin::Foo                 # [Foo]                 dist-zilla plugins for general use
Dist::Zilla::Plugin::Author::KENTNL::Foo # [Author::KENTNL::Foo] foo that only KENTNL will probably have use for
Dist::Zilla::PluginBundle::Classic       # [@Classic]            A bundle that can have practical use by many
Dist::Zilla::PluginBundle::Author::KENTNL #[@Author::KENTNL]     KENTNL's primary plugin bundle
Dist::Zilla::MintingProfile::Default     # A minting profile that is used by all
Dist::Zilla::MintingProfile::Author::KENTNL # A minting profile that only KENTNL will find of use.

Making Dist::Zilla::MintingProfile::Author::YOURID

First, we'll start with the above default empty profile with a basic dist.ini

$ dzil new Dist-Zilla-MintingProfile-Author-YOURID
And then the guts of that Minting Profile is pretty basic.
use strict;
use warnings;

package Dist::Zilla::MintingProfile::Author::YOURID;

# ABSTRACT: YOURID's Minting Profile

use Moose;
use namespace::autoclean;
with 'Dist::Zilla::Role::MintingProfile::ShareDir';


__PACKAGE__->meta->make_immutable;
no Moose;
1;

This is a deceptively simple piece of magic that took me a little while to fully grok.

Next, we make a 'profile' directory in our dist.

$ mkdir -p Dist-Zilla-MintingProfile-Author-YOURID/share/profiles/
And in that, we can put one or many profiles. For simplicity, we'll just start off with a 'default' profile.
$ mkdir -p Dist-Zilla-MintingProfile-Author-YOURID/share/profiles/default/
$ pushd Dist-Zilla-MintingProfile-Author-YOURID/share/profiles/default/

Now, you could just stick the same profile.ini you had above in ~/.dzil there, and you're mostly done with the hard part. However, I'll go a little more into detail so you get an idea of what use it is.

Make a 'skel' dir

Make a 'skel' dir holding a selection of templated-files you want in all new dists

Here is the most notable ones in mine

$ mkdir skel
# stuff directory with files
$ find skel/
# skel/
# skel/.gitignore
# skel/.perltidyrc
# skel/Changes
# skel/perlcritic.rc
# skel/weaver.ini
$ cat skel/Changes
# Release history for {{ $dist->name }}
# 
# {{ '{{$NEXT}}' }}
#
#        First version, released on an unsuspecting world.
#
$ cat skel/.gitignore
# .build
# {{ $dist->name }}-*

{{ $dist->name }} is expanded during 'new' to the name of the intended dist, and {{ '{{$NEXT}}' }} is just a nasty hack so that the literal string {{$NEXT}} turns up in the changelog file so the Changelog plugin keeps working.

For more in-depth understanding, please read The dzil minting profile tutorial

Adding support for the skel directory

You now need to tell your profile.ini about that skel directory if you want it to mean anything.

[GatherDir::Template]
root = skel
include_dotfiles = 1 ; I want the dot files! 
./profile.ini    # this file ---> indicates inclusion of
./skel                           #      /
./skel/.gitignore          <-----------/
./skel/.perltidyrc         <----------/
./skel/Changes             <---------/
./skel/perlcritic.rc       <--------/
./skel/weaver.ini          <-------/

Other misc profile additions

Now most people will probably just put in
./skel/dist.ini

With some template variables expanded, but I myself prefer the dedicated module approach for dist.ini, so I add that to profile.ini

[GatherDir::Template]
root = skel
include_dotfiles = 1 ; I want the dot files! 

[Author::KENTNL::DistINI]
And for good measure, I want every new dist to automatically have git set up with it, so I add that plugin too.
[GatherDir::Template]
root = skel
include_dotfiles = 1 ; I want the dot files! 

[Author::KENTNL::DistINI]
[Git::Init]

Gluing it together

Now if you've been following me, you'll have a directory that looks a lot like this:
dist.ini
lib/
lib/Dist
lib/Dist/Zilla
lib/Dist/Zilla/MintingProfile
lib/Dist/Zilla/MintingProfile/Author
lib/Dist/Zilla/MintingProfile/Author/YOURID.pm
share/
share/profiles
share/profiles/default
share/profiles/default/profile.ini
share/profiles/default/skel
share/profiles/default/skel/.gitignore
share/profiles/default/skel/.perltidyrc
share/profiles/default/skel/Changes
share/profiles/default/skel/perlcritic.rc
share/profiles/default/skel/weaver.ini

And you're probably wondering how the 'share' directory and its contents will be seen by 'YOURID.pm' after installation.

The magic is this stanza in your dist.ini

[ModuleShareDirs]
Dist::Zilla::MintingProfile::Author::YOURID = share/profiles 

Yeah, at first that baffled me too.

Into the guts of File::ShareDir

Dist::Zilla utilizes sharedir in 2 places, 1. In the module that does the minting, and 2. Using some instructions injected in your install tool.

During Install

Lets pretend there is a directory, we'll call it $x, which is known as the "base directory" for all Things File::ShareDir::Install installs, which is also known to to File::ShareDir.

For example:

/usr/lib64/perl5/vendor_perl/5.12.2/auto/share/    # $x 
/usr/lib64/perl5/vendor_perl/5.12.2/               # module install dir.

The ModuleShareDirs plugin tells File::ShareDir::Install that the directory 'share/profiles' should be installed in a path associated with the module, i.e.:

$x/module/Dist-Zilla-MintingProfile-Author-YOURID/ <= share/profiles/ 

So, during install, the files in share/profiles/* will be copied to that directory.

.../auto/share/module/Dist-Zilla-MintingProfile-Author-YOURID/default/skel/perlcritic.rc
.../auto/share/module/Dist-Zilla-MintingProfile-Author-YOURID/default/skel/weaver.ini
.../auto/share/module/Dist-Zilla-MintingProfile-Author-YOURID/default/skel/.perltidyrc
.../auto/share/module/Dist-Zilla-MintingProfile-Author-YOURID/default/skel/.gitignore
.../auto/share/module/Dist-Zilla-MintingProfile-Author-YOURID/default/skel/Changes
.../auto/share/module/Dist-Zilla-MintingProfile-Author-YOURID/default/profile.ini
.../Dist/Zilla/MintingProfile/Author/YOURID.pm

And that is not all that scary =)

How your profile uses that

Having seen how it installs to the file-system, how the plug-in works should be a bit of a no brainer.

  • dzil new -P Author::YOURID -p default Some-Module-Name
  • Dist::Zilla, sees -P, and loads the respective profile from @INC, Dist::Zilla::MintingProfile::Author::YOURID
  • Your minting profile uses that sharedir role, and asks File::ShareDir for the directory associated with 'Dist-Zilla-MintingProfile-Author-YOURID', which of course returns that '.../auto/share/module/Dist-Zilla-MintingProfile-Author-YOURID/' we installed above.
  • The Role then determines which sub-profile to use ( default, as specified by -p ) and then returns that path ( now .../auto/share/module/Dist-Zilla-MintingProfile-Author-YOURID/default/ )
  • .../auto/share/module/Dist-Zilla-MintingProfile-Author-YOURID/default/profile.ini is then read and a new distribution is constructed using the instructions therein
  • And presto, you have a newly minted dist =):
    $ cd Some-Module-Name;
    $ find
    # ./.git 
    # ... a bunch of .git files ...
    # ./dist.ini
    # ./weaver.ini
    # ./perlcritic.rc
    # ./Changes
    # ./.perltidyrc
    # ./.gitignore
    # ./lib
    # ./lib/Some
    # ./lib/Some/Module
    # ./lib/Some/Module/Name.pm
    

Hopefully thats enough to get you started, but...

There is one warning you must uptake with the use of file sharedir.

ONCE YOU INSTALL A FILE IN A SHAREDIR FROM A CPAN DIST, IT IS THERE FOR GOOD

At least, it will be there till somebody manually deletes it, at least in the current implementation of CPAN.

This means if you decide to put a file in skel/, and then later decide you don't want that file in skel/, you're going to have a problem when something decides to "add all of skel/ to your new dist".

For vendor-wrapped/packaged stuff ( ie: Redhat, Debian, Gentoo ) you're fine, those systems have package management that guarantee atomicity somewhat, but the CPAN toolchain ( at least, in my experience ) provides very little guarantee in this regard. This may change in the future, but that's how it seems to be now. But now I've warned you, you'll hopefully not bump into that =).

5 comments:

  1. How about Author::$AUTHOR::Dist::Zilla::MintingProfile

    it could just be a whole ignorable namespace. and it's a lot easier to determine where to put $AUTHOR if you have several modules that you need your own namespace for.

    ReplyDelete
  2. also, in all of this, where are the tests? this doesn't have any tests?

    ReplyDelete
  3. With plugins, its often the case that using Author::$AUTHOR as the root namespace ( from here to be labelled as simply $AUTHTOKEN ) is very annoyingly impractical.

    Most plugin-systems tend to want plugins to be sub-spaced of their own namespace, and use assumed prefixing to make developers lives easier.

    ie: dzil new -P Foo -> Dist::Zilla::MintingProfile::Foo

    Handling this when $AUTHTOKEN is at the start is one thing, but it gets worse when you look at how people would use that in plugins:

    $AUTHTOKEN::Dist::Zilla::Plugin::DistINI <-- horrible.

    [=$AUTHTOKEN:Dist::Zilla::Plugin::DistINI] vs
    [$AUTHTOKEN::DistINI] <-- wins.

    You could possibly hijack it with some magic syntax, ie:

    [$AUTHTOKEN => DistINI] # $AUTHTOKEN:Dist::Zilla::Plugin::DistINI
    [$AUTHTOKEN => @Bundle] # AUTHTOKEN:Dist::Zilla::PluginBundle::Bundle

    But thats starting to look like a finding a problem for a solution.

    As for tests, well, thats why its on CPAN, so you can submit patches ;)

    ReplyDelete
  4. I would never submit patches to something in $AUTHOR, mostly because I figure it's your own thing and you have it how you want it, it's not really for me to judge, or really use.

    ReplyDelete
  5. My stance on that is: Even then, theres a difference between "bug" and "by design", If it seems like a reasonable bug, I'd file it anyway and just be aware it might get rejected.

    DZPB::RJBS : 1 Bug pending https://rt.cpan.org/Public/Dist/Display.html?Status=Active&Name=Dist-Zilla-PluginBundle-RJBS

    =P.

    ReplyDelete