tldr: Proposal: Change loading packages from pack/start (and
:packloadall) to first add all directories to 'runtimepath' before
sourcing the plugins.


Lcd wrote:

> > > > > On 2 April 2016, Bram Moolenaar <b...@moolenaar.net> wrote:
> > > > > > 
> > > > > > Patch 7.4.1699
> > > > > > Problem:    :packadd does not work the same when used early or late.
> > > > > > Solution:   Always load plugins matching "plugin/**/*.vim".
> > > > > > Files:      src/ex_cmds2.c, src/testdir/test_packadd.vim
> > > > > > 
> > > > > > 
> > > > > > *** ../vim-7.4.1698/src/ex_cmds2.c  2016-03-19 14:16:34.611690741 
> > > > > > +0100
> > > > > > --- src/ex_cmds2.c  2016-04-02 22:29:44.513040078 +0200
> > > > > > ***************
> > > > > > *** 3376,3382 ****
> > > > > >   
> > > > > >       if (load_files)
> > > > > >       {
> > > > > > !   static char *plugpat = "%s/plugin/*.vim";
> > > > > >     static char *ftpat = "%s/ftdetect/*.vim";
> > > > > >     int         len;
> > > > > >     char_u      *pat;
> > > > > > --- 3376,3382 ----
> > > > > >   
> > > > > >       if (load_files)
> > > > > >       {
> > > > > > !   static char *plugpat = "%s/plugin/**/*.vim";
> > > > > >     static char *ftpat = "%s/ftdetect/*.vim";
> > > > > >     int         len;
> > > > > >     char_u      *pat;
> > > > > [...]
> > > > > 
> > > > >     I'm afraid this is not a good idea.  Wildcards are expanded depth
> > > > > first, which means dependencies are now loaded before the main script.
> > > > > This breaks scripts that are themselves designed to use plugins, e.g.:
> > > > > 
> > > > > Error detected while processing 
> > > > > /home/lcd047/Git/Vim/textobj-comment/plugin/textobj/comment.vim:
> > > > > line   27:
> > > > > E117: Unknown function: textobj#user#plugin
> > > > > Press ENTER or type command to continue
> > > > > 
> > > > >     I believe this seemingly unrelated report is likely to be another
> > > > > symptom of the same problem:
> > > > > 
> > > > > On 3 April 2016, Naruhiko Nishino <naru123456...@gmail.com> wrote:
> > > > > > Hi Bram,
> > > > > > 
> > > > > > I found a case below when using `:packloadall` in .vimrc.
> > > > > > 
> > > > > >   A/plugin/a.vim -- call B's autoload functions.
> > > > > >   B/plugin/b.vim
> > > > > > 
> > > > > > I think A can laod B's autoload functions.
> > > > > > Do you think this?
> > > > > [...]
> > > > 
> > > > Hmm, plugins were always loaded this way.  See main.c:
> > > > 
> > > >         source_runtime((char_u *)"plugin/**/*.vim", DIP_ALL);
> > > > 
> > > > So, why did this work fine before but not now?
> > > 
> > >     "textobj-comment" in my example above is this package:
> > > 
> > >         https://github.com/glts/vim-textobj-comment
> > > 
> > >     It doesn't do anything by itself, it's just a plugin for
> > > "textobj-user", which itself is a framework that can load specially
> > > crafted plugins:
> > > 
> > >         https://github.com/kana/vim-textobj-user
> > > 
> > >     "textobj-comment" doesn't need, nor expects, to be run from anything
> > > else than "textobj-user".  Its structure looks like this:
> > > 
> > >         textobj-comment
> > >         |-- Makefile
> > >         |-- README.markdown
> > >         |-- VimFlavor
> > >         |-- addon-info.json
> > >         |-- autoload
> > >         |   `-- textobj
> > >         |       `-- comment.vim
> > >         |-- doc
> > >         |   |-- tags
> > >         |   `-- textobj-comment.txt
> > >         |-- plugin
> > >         |   `-- textobj
> > >         |       `-- comment.vim
> > >         `-- t
> > >             |-- fixtures
> > >             |   |-- Inline.java
> > >             |   |-- paired.c
> > >             |   `-- simple.py
> > >             |-- inline.vim
> > >             |-- leaders.vim
> > >             |-- paired.vim
> > >             |-- plugin.vim
> > >             |-- simple.vim
> > >             `-- util
> > >                 `-- helpers.vim
> > 
> > Well, that is confusing. vim-textobj-user calls itself a plugin, but
> > it doesn't have a plugin file.
> 
>     It is indeed confusing.  First, the word "plugin" is overloaded
> above.  For the purpose of this discussion, let's refer to Vim plugins
> as "packages", and save "plugins" for the more general concept of
> "modules loaded on-demand by something else".
> 
> > There is no mention of how to install it, I assume one has to trop
> > the "autoload/textobj/user.vim into ~/.vim/autoload/textobj/user.vim.
> > Thus it's to be invoked from other plugins, functions as a kind of
> > library.
> >
> > So vim-textobj-comment is a plugin, and its plugin/textobj/comment.vim
> > must be sourced for it to become active.  It depends on
> > vim-textobj-user to exist, but of course the user must make sure it
> > can be found, thus it must be in 'runtimepath'.
> 
>     No.  "vim-textobj-user" is a framework for extending text objects.
> It does nothing by itself, but it can load third-party plugins to create
> new text objects.  Various Vim packages are available, that provide
> these plugins.  "vim-textobj-comment" is such a package, that adds text
> objects for comments.

The thing is that vim-textobj-user does not load anything.  It is to be
loaded by those other plugins, they depend on it.

>     "vim-textobj-user" can make sense without "vim-textobj-comment", if
> other plugins are installed that provide text objects for it.
> 
>     "vim-textobj-comment" needs "vim-textobj-user" to do anything
> useful, and it doesn't make sense without it.  You can think of
> "vim-textobj-comment" as a package and "vim-textobj-user" as a library,
> but this is somewhat misleading because the main functionality is still
> provided by "vim-textobj-user".  It's like calling the UNIX kernel a
> library for the USB mass storage layer.  "vim-textobj-comment" is just a
>       plugin for "vim-textobj-user".

What matters is that vim-textobj-comment depends on vim-textobj-user,
and that last one must be in 'rumtimepath' before loading the first one.

> > It has a command for that for NeoBundle:
> > 
> >     if exists(':NeoBundleDepends') == 2
> >       NeoBundleDepends 'kana/vim-textobj-user'
> >     endif
> > 
> > So this only works with NeoBundle.
> 
>     It works with any other manager.  I'd say the above piece of code
> exists because NeoBundle is the only manager that insists to source
> things deeper than "plugin/*.vim".  Others are happy with just adding
> "textobj-comment" to runtimepath.  Looking at the sources, actually
> loading scripts deeper than the first layer is generally left as an
> option, and is off by default.

There is a dependency here on how that plugin manager works.  It must
add directories to 'runtimepath' before loading any plugin.  It appears
the real problem is that :packadd only adds the plugin that you give it,
and none of its dependencies.

I have been staying away of dependency management, since that is more
the task of plugin managers.  On the other hand, having the user figure
out dependencies and specify them is not what we want.  Ideally, the
plugin writers add :packadd commands to their plugin to load
dependencies where needed.

So yes, that requires the plugin author to be aware of packages and
modify his plugins to use them.  What appears to be missing here is
that a user would want to download some plugins and drop them in a
directory under pack/start and expect everything to work.

Perhaps there is one solution that is the middle road: When loading all
plugins under pack/start, first add all directories to 'runtimepath'
before sourcing any plugins.  I can't think of a reason why that would
break for who is using the current behavior, and it does take away the
dependency on the order in which plugins are found and loaded.


> [...]
> > >     Now, in my experience most plugins expect Vim to source only
> > > "plugin/*.vim", not "plugin/**.vim".  But that's only my experience,
> > > and I'm obviously only familiar to a tiny minority of plugins
> > > compared to what exists out there.  But I still think "plugin/*.vim"
> > > makes more sense. *shrug*
> >
> > Again, no, Vim has always sourced plugin/**/*.vim.  The
> > vim-textobj-comment also depends on this, otherwise it would never
> > have plugin/textobj/comment.vim loaded.
> 
>     There is still the issue of finding scripts depth-first.  This is
> fine for glob(), and possibly also for runtimepath, but it doesn't make
> sense for loading.  Presumably the main script always has to be loaded
> first.

I haven't seen an example of this, and as mentioned, it already is the
current behavior.

-- 
'Psychologist' -- Someone who looks at everyone else when
an attractive woman enters the room.

 /// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net   \\\
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\  an exciting new programming language -- http://www.Zimbu.org        ///
 \\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

-- 
-- 
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups 
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Raspunde prin e-mail lui