review, multiple concerns

[ikiwiki] / doc / todo / allow_site-wide_meta_definitions.mdwn

[[!tag plugins/meta patch]]
[[!template id=gitbranch branch=jon/defaultmeta author="[[Jon]]"]]

I'd like to define [[plugins/meta]] values to apply across all pages
site-wide unless the pages define their own: default values for meta
definitions essentially.

    <snip old patch, see below for latest>

-- [[Jon]]

> This doesn't support multiple-argument meta directives like
> `link=x rel=y`, or meta directives with special side-effects like
> `updated`.
>
> The first could be solved (if you care) by a syntax like this:
>
>     meta_defaults => [
>         { copyright => "© me" },
>         { link => "about:blank", rel => "silly", },
>     ]
>
> The second could perhaps be solved by invoking `meta::preprocess` from within
> `scan` (which might be a simplification anyway), although this is complicated
> by the fact that some (but not all!) meta headers are idempotent.
> 
> --[[smcv]]

>> Thanks for your comment. I've revised the patch to use the config syntax
>> you suggest. I need to perform some more testing to make sure I've
>> addressed the issues you highlight.
>> 
>> I had to patch part of IkiWiki core, the merge routine in Setup, because
>> the use of `possibly_foolish_untaint` was causing the hashrefs at the deep
>> end of the data structure to be converted into strings. The specific change
>> I've made may not be acceptable, though -- I'd appreciate someone providing
>> some feedback on that hunk!

>>> Well, re that hunk, taint checking is currently disabled, but
>>> if the perl bug that disallows it is fixed and it is turned back on,
>>> the hash values will remain tainted, which will probably lead to
>>> problems.
>>>
>>> I'm also leery of using such a complex data structure in config.
>>> The websetup plugin would be hard pressed to provide a UI for such a
>>> data structure. (It lacks even UI for a single hash ref yet, let alone
>>> a list.)
>>> 
>>> Also, it seems sorta wrong to have two so very different syntaxes to 
>>> represent the same meta data. A user without a lot of experience will
>>> be hard pressed to map from a directive to this in the setup file.
>>>
>>> All of which leads me to think the setup file could just contain
>>> a text that could hold meta directives. Which generalizes really to
>>> a text that contains any directives, and is, perhaps appended to the
>>> top of every page. Which nearly generalizes to the sidebar plugin,
>>> or perhaps something more general than that... 
>>>
>>> However, excessive generalization is the root of all evil, so 
>>> I'm not necessarily saying that's a good idea. Indeed, my memory
>>> concerns below invalidate this idea pretty well. --[[Joey]] 

    diff --git a/IkiWiki/Plugin/meta.pm b/IkiWiki/Plugin/meta.pm
    index 6fe9cda..2f8c098 100644
    --- a/IkiWiki/Plugin/meta.pm
    +++ b/IkiWiki/Plugin/meta.pm
    @@ -13,6 +13,8 @@ sub import {
        hook(type => "needsbuild", id => "meta", call => \&needsbuild);
        hook(type => "preprocess", id => "meta", call => \&preprocess, scan => 1);
        hook(type => "pagetemplate", id => "meta", call => \&pagetemplate);
    +   hook(type => "scan", id => "meta", call => \&scan)
    +           if $config{"meta_defaults"};
     }
     
     sub getsetup () {
    @@ -305,6 +307,15 @@ sub match {
        }
     }
     
    +sub scan() {
    +   my %params = @_;
    +   my $page = $params{page};
    +   foreach my $default (@{$config{"meta_defaults"}}) {
    +           preprocess(%$default, page => $page,
    +                   destpage => $page, preview => 0);
    +   }
    +}
    +
     package IkiWiki::PageSpec;
     
     sub match_title ($$;@) {
    diff --git a/IkiWiki/Setup.pm b/IkiWiki/Setup.pm
    index 8a25ecc..e4d50c9 100644
    --- a/IkiWiki/Setup.pm
    +++ b/IkiWiki/Setup.pm
    @@ -51,7 +51,13 @@ sub merge ($) {
                                        $config{$c}=$setup{$c};
                                }
                                else {
    -                                   $config{$c}=[map { IkiWiki::possibly_foolish_untaint($_) } @{$setup{$c}}]
    +                                   $config{$c}=[map {
    +                                           if(ref $_ eq 'HASH') {
    +                                                   $_
    +                                           } else {
    +                                                   IkiWiki::possibly_foolish_untaint($_)
    +                                           }
    +                                   } @{$setup{$c}}];
                                }
                        }
                        elsif (ref $setup{$c} eq 'HASH') {
    diff --git a/doc/ikiwiki/directive/meta.mdwn b/doc/ikiwiki/directive/meta.mdwn
    index 000f461..8d34ee4 100644
    --- a/doc/ikiwiki/directive/meta.mdwn
    +++ b/doc/ikiwiki/directive/meta.mdwn
    @@ -12,6 +12,16 @@ also specifies some additional sub-parameters.
     The field values are treated as HTML entity-escaped text, so you can include
     a quote in the text by writing `&quot;` and so on.
     
    +You can also define site-wide defaults for meta values by including them
    +in your setup file. The key used is `meta_defaults` and the value is a list
    +of hashes, one per meta directive. e.g.:
    +
    +   meta_defaults = [
    +           { copyright => "Copyright 2007 by Joey Hess" },
    +           { license   => "GPL v2+" },
    +           { link => "somepage", rel => "site entrypoint", },
    +   ],
    +
     Supported fields:
     
     * title

-- [[Jon]]

>> Ok, I've had a bit of a think about this. There are currently 15 supported
>> meta fields. Of these: title, licence, copyright, author, authorurl,
>> and robots might make sense to define globally and override on a per-page
>> basis.
>> 
>> Less so, description (due to its impact on map); openid (why would
>> someone want more than one URI to act as an openid endpoint to the same
>> place?); updated. I can almost see why someone might want to set a global
>> updated value. Almost.
>> 
>> Not useful are permalink, date, stylesheet (you already have a global
>> stylesheet), link, redir, and guid.
>>
>> In other words, the limitations of my first patch that [[smcv]] outlined
>> are only relevant to defined fields that you wouldn't want to specify a
>> global default for anyway.
>>
>>> I generally agree with this. It is *possible* that meta would have a new
>>> field added, that takes parameters and make sense to use globally.
>>> --[[Joey]] 
>>
>> Due to this, and the added complexity of the second patch (having to adjust
>> `IkiWiki/Setup.pm`), I think the first patch makes more sense. I've thus
>> reverted to it here.
>>
>> Is this merge-worthy?

    diff --git a/IkiWiki/Plugin/meta.pm b/IkiWiki/Plugin/meta.pm
    index b229592..3132257 100644
    --- a/IkiWiki/Plugin/meta.pm
    +++ b/IkiWiki/Plugin/meta.pm
    @@ -13,6 +13,7 @@ sub import {
        hook(type => "needsbuild", id => "meta", call => \&needsbuild);
        hook(type => "preprocess", id => "meta", call => \&preprocess, scan => 1);
        hook(type => "pagetemplate", id => "meta", call => \&pagetemplate);
    +   hook(type => "scan", id => "meta", call => \&scan);
     }
     
     sub getsetup () {
    @@ -302,6 +303,15 @@ sub match {
        }
     }
     
    +sub scan() {
    +   my %params = @_;
    +   my $page = $params{page};
    +    foreach my $type (map { s/^meta_//; $_ } grep /^meta_/, keys %config) {
    +           $pagestate{$page}{meta}{$type} = $config{"meta_$type"}
    +                   unless defined $pagestate{$page}{meta}{$type};
    +   }
    +}
    +
     package IkiWiki::PageSpec;
     
     sub match_title ($$;@) {
    diff --git a/doc/ikiwiki/directive/meta.mdwn b/doc/ikiwiki/directive/meta.mdwn
    index 000f461..200c4b2 100644
    --- a/doc/ikiwiki/directive/meta.mdwn
    +++ b/doc/ikiwiki/directive/meta.mdwn
    @@ -12,6 +12,12 @@ also specifies some additional sub-parameters.
     The field values are treated as HTML entity-escaped text, so you can include
     a quote in the text by writing `&quot;` and so on.
     
    +You can also define site-wide defaults for meta values by including them
    +in your setup file, e.g.
    +
    +   meta_copyright => "Copyright 2007 by Joey Hess",
    +   meta_license   => "GPL v2+",
    +
     Supported fields:
     
     * title

-- [[Jon]]

>>> Merry Christmas/festive season/happy new year folks. I've been away from
>>> ikiwiki for the break, and now I've returned to watching recentchanges.
>>> Hopefully I'll be back in the mix soon, too. In the meantime, Joey, have
>>> you had a chance to look at this yet? -- [[Jon]]

>>> For this to work with websetup and --dumpsetup, it needs to define the
>>> `meta_*` settings in the getsetup function.
>>>
>>> I also have some concerns about both these patches, since both throw
>>> a lot of redundant data at meta, which then stores it in a very redundant
>>> way. Specifically, meta populates a per-page `%metaheaders` hash
>>> as well as storing per-page metadata in `%pagestate`. So, if you have
>>> a wiki with a thousand pages, and you add a 1k site-wide license text,
>>> that will bloat the memory usage of ikiwiki by in excess of 2
>>> megabytes. It will also cause ikiwiki to write a similar amount more data
>>> to its state file which has to be loaded back in each
>>> run.
>>>
>>> Seems that this could be managed much more efficiently by having
>>> meta special-case the site-wide settings, not store them in these
>>> per-page data structures, and just make them be used if no per-page
>>> metadata of the given type is present. --[[Joey]]