FIXlib
Table of Contents
1 Introduction
This paper is part of my Commonplace Book; it descripes how I use local fixlibs.
A fixlib is local to a directory. It's first job is to collect local functions needing repair. It may be the first place a function is stored.
$ aNewfunction () { ... ; }
$ declare -f aNewfunction | tee -a fixlib
The fixlib may be edited by your favorite editor. Occasionally, it is prudent to back it up. Here is the only backup you'll ever need.
$ backuplib fixlib
The preferred home for local functions is in a library named for its
intended purpose. One of my common tasks these days, being "retired"
is to manage our family investments. So in a HOME/Family/invest
directory you'll find an investlib. When functions in the local
fixlib achieve a certain stability ( I haven't found the need to
repair or extend them), then they should be transferred to the
investlib.
1.1 function usage
That's the first step on the way to a function practice. There are two other destinations for a function: frequent use on the command line, a small utility, and a tool for repeated use in other functions. Of these latter, I'm also discovering two classes of function: those which support shell function development in general, and those which support another tool which may or not be used in any local application. That function tree looks like:
- local functions
- global functions
- command line utilities
- widely used functions
- function development practice
- other supporting tools
In the remainder of this paper, let's look at the various path's a function may take. Before jumping off, it's necessary the description to this point is the ideal. There are likely many fixlib in your account at any time, and it's possible there's a global function which may have been pulled down into more than one fixlib. So, we'll include a few functions to identify where these might be.
How does it happen, that a function may be in multiple places at once? There's noting from preventing a quick fix happening in any working directory:
declare -f someGlobalFunction | tee -a fixlib
Some time later, after hopefully upward-compatible fixes to the function, you can find out where it belongs to put it back:
ifh someGlobalFunction
This is the first function we've seen here in the examples. Here are it's component parts:
1.2 supporting code
ihf ()
{
: ~ function
: reports first instance of linked library names
quietly whf $1 | xargs ls -i | awk ' !p[$1] ++ { print $2 }'
}
whf ()
{
: ~ function
: reports names of function libraries where function is defined
: USES funclocn -- returns the list of active function libraries
echo $* 1>&2;
xwh "$1" $(funclocn) | sed 's/ .*//; s/:.*//' | tr A-Z a-z | uniq
}
quietly ()
{
: ~ command ... -- gobbles stderr from command
$@ 2> /dev/null
}
xwh ()
{
patn="$1";
shift;
echo $# ... $1 $2 $3 ... 1>&2;
grep "^$patn[[:blank:]][[:blank:]]*()" $*;
grep "^function $patn *#*.*$" $*
}
1.3 use example
$ quietly foreach do_whf whf ihf xwh whf ./fixlib /users/applemcg/.profile ihf ./fixlib /users/applemcg/dropbox/git/backash/bin/cmdlib xwh /users/applemcg/.profile
From this, we see that the whf function is defined in two places, the local fixlib and my home .profile. That means, if the function is updated in the fixlib, it should be returned to the .profile. We will have to decide, in a separate exercise, if there are other intances of whf out for a fix. Similarly for ihf. However xwh is only found in the .profile.
For the moment, we won't consider that any of these functions may be in another fixlib.
If we can guarantee that functions in this local fixlib aren't defined in another fixlib, then regardless of whether or not they've been changed here, then the copies here, disregarding other copies may be returned to the proper library. Here's the first example:
source fixlib declare -f whf | tee -a /users/applemcg/.profile
After repeating the process for each function, they may be removed from the fixlib.
functions fixlib | egrep -v '^(whf|ihf|xwh)$' > .x . fixlib; declare -f $(< .x) | tee .l mv .l fixlib
2 Capturing functions
Following the outline of the various uses of a function, here's how to capture them in their proper resting place. Recall this may change as the function becomes more widely useful.
2.1 locally
We've already seen this but let's refesh it here for completeness:
declare -f newFunction | tee -a fixlib
The question arises: how to avoid multiple copies of the same function? We'll answer this after the outline.
2.2 globally
Recall, there are two classes here as well: a command line utility or a function widely used as sub-function. First, the command line:
declare -f cmdLineUtility | tee -a $(which cmdlib)
In this case, I maintain a library cmdlib, stored on my PATH, which is included
by a source cmdlib in my .profile.
Alternatively, a function may be destined for a widely-used shell programming feature or as an addition to a existing publicly available tool. Since any of these are also found on my path, storing them is the same, with the exception of the library name.
2.2.1 programming
In practice, I actually have two of these. One called auxlib, the other known as programlib. The auxlib is more rigorously controlled, the programlib, is an alternate to the cmdlib, but assigned the same way.
declare -f report_feature | tee -a $(which programlib)
There's a pair of functions to identify where a function is used:
sfuse and Set Functions USE, and fuse, just Functions USE, the
latter reading from stdin. The former, sfuse displays the results
of the set command, where all the function definitions and shell
varialbes are displayed.
$ declare -f sfuse fuse
sfuse ()
{
trace_call $*;
set | fuse "$1"
}
fuse ()
{
report_notargcount 1 $# pattern && return 1;
: reconcile patterns to fun_from;
p="$1";
shift;
cat ${*:--} | awk "
\$1 ~ /^:$/ || \$1 ~ /^comment$/ { next }
\$2 ~ /^\(\)$/ { f = \$1 }
/^function / { f = \$2 }
f != \"\" && ( \
/[^a-zA-Z0-9_-]$p[^a-zA-Z0-9_-]/ || \
/[^a-zA-Z0-9_-]$p$/ ) {
printf \"%s\t%s\n\", f, \$0
}
"
}
$ sfuse fuse
fun_create fuse eval | grep '()' | sed 's/ *()/ ()/; s/ *\$/$/' | awk '$4 ~ /()/'
fun_maker set | fuse eval | grep ' () ' | field 1
nuse set | fuse $1 | grep -v ' () *$' | wc -l
sfuse set | fuse "$1"
So, the later usage: sfuse fuse shows the list of functions and the
context for each instance of the fuse function. In the current
scope, this shows that fuse is used by fun_create, fun_maker,
nuse, and sfuse itself.
What does this tell us? That fuse belongs in programlib and not cmdlib
2.2.2 tool interfaces
It turns out, that sfuse is primarily a cmdlib tool, since I use it primarily on the command line. But it also nicely shows an exception to the rule which introduces the final component.
$ ff fun_bash obs_names
fun_bash ()
{
report_notargcount 1 $# function name && return 1;
trace_call $*;
rdb_hdr bash context;
sfuse $1
}
obs_names ()
{
trace_call $*;
sfuse OBSOLESCENT | field 1
}
$ fun_bash sfuse
bash context
---- -------
fun_bash sfuse $1
obs_names sfuse OBSOLESCENT | field 1
The exception here is that sfuse is indeed used in other functions. How did we find these? We used one of them, fun_bash to display the connection to another tool, in this case, the underutilized /RDB. In this case I used it to produce a small sample table of the sfuse output, folded into an /rdb table, where the field names are bash and context, listing the shell output in the table.
A partial list of tools with wrapper libraries:
- git
- github command line
- mark
- Markdown
- pan
- pandoc, swiss-army knife of text formatters
- rdb
- the Relational Data Base.
- t
- the twitter API
- yaib
- yahoo API lib
2.3 common practice
This top-level set of functions uses copies of functions
found in the local fixlib in common with the current run-time
version of the function: lib = which $1. In fun_toliblist,
the functions are first copied to the existing library, and then,
in fun_topreflib are removed from the default fixlib.
fun_golib ()
{
: ~ libraryName
: functions in fixlib are returned to their home library
: and culled from the fixlib
: USES report_ , functions, fun_starter, backup_lib, and flcomm
local lib=$(which $1); shift
report_notfile $lib /dev/null && return 1
local fix=./fixlib
local tmp=./.l
: functions in BOTH in FIX and LIB, "comm -12 .."
local lst=$(flcomm -12 $fix $lib)
[[ -z "$lst" ]] && {
comment no FIXLIB functions to $lib
comment ===========================
quietly foreach do_whf $(functions $fix)
return
}
. $fix # source FIXLIB, the FIXed versions
fun_toliblist $lib $lst
comment =================
fix_whf
}
fun_toliblist ()
{
: ~ library function ...
: append functions to library, back it up
# report_notcalledby fix_golib && return 1
lib=$1; shift
tmp=./.l
shell_write $tmp declare -f $(functions $lib) $*
shell_append $tmp fun_starter $lib
mv $tmp $lib
backup_lib $lib
fun_preferlib $lib
}
fun_preferlib ()
{
: ~ library [fixlib]
: functions unique to 2nd arg are retained,
: library is NOT updated
report_notfile $1 && return 1
lib=$1; shift
frm=${2:-./fixlib}; shift
report_notfile $frm && return 2
tmp=./.l
shell_write $tmp declare -f $(flcomm -23 $frm $lib)
doit mv $tmp $frm
backup_lib $frm
}
fun_whf () { do_do whf; quietly foreach do_whf $(functions fixlib); }
fix_only ()
{
fun_whf | awk 'NF == 2 { print $1 }'
}