A Parser Versus a Programmer's Front End | Missing and Planned Features | The 80/20 Rule | Comparing Ease of Use | Category of Command Line Usage Patterns | Ideas for OptiFlag functionality (Flag Set Constants) | Ideas for OptiFlag functionality (Argument) | Ideas for OptiFlag functionality (Flag clustering) | Ideas for OptiFlag functionality (Sub Commands) |


A Parser Versus a Programmer's Front End

A Parser Versus a Programmer's Front End I debated calling OptiFlag a command-line parser. First, the term parser implies a terrible amount of work, and an even more terrible knowledge of computer science courses that I never had the priveledge of taking. (Phrases like "context-free grammmar" float in my head). Second, and more appropriate at this time, is the fact that I do not want to reimplement the parsing code that has been written thousands of times before, mainly because I do not think I could do a better job at it.

Nevertheless, as soon as I started OptiFlag for other reasons (ease-of-use), I found myself implementing a basic parser -- do not understand "parser" to be anything more than several hacked-together methods that scan ARGV looking for patterns. Upon reflection, and especially after hitting release 0.6.5, I realized that my parsing code would be unsustainable. Each time I used another command-line program, I discovered more unforseen usage scenarios. And each time I tried to add the capability to the OptiFlag parser code, I found that the bloat continued to grow.

So why do I mention all of this? Am I saying that OptiFlag has no reason to exist, no motivation for its upkeep? Of course not. But the question and its answer have made it clear what OptiFlag's primary purpose is, and why I created it in the first place: to make life easier for the programmer. Hopefully, I will have created a mini-DSL library that is more in line with the "ruby way".

For the near future, OptiFlag will continue to maintain its own parsing code. It is remarkable what I have been able to accomplish using simple array manipulation algorithms (ARGV is an array, after all). But if I have ever have to accomplish the full set of functionality that another program, such as Getopt::Declare accomplishes, I will be setting myself up for a long haul.

It is more appropriate to consider OptiFlag a Programmer's Front End. It is now, and will be later, a front end to the parsing code that does the actual work. It is the syntax of a programming language compared to its compiler -- mayhap, I take solace in the fact that Ruby itself is an excellently designed programming language with a reputation for a less than stellar implementation (I guess I can get away with saying this as Matz himself has mentioned it plenty of times).

It would be my hope that in the future, I will be able to swap in a parser for a pre-existing command-line library transparently to the user. Ideally, it would be one of the command-line libraries that comes pre-installed with the Ruby installation (like getoplong or getopt) so as not to introduce too many dependencies and maintenance issues -- although, my current favorite is the external library Getopt::Declare which has a completely different means of declaration and usage ( a legacy it gained from PERL).

Missing and Planned Features

Missing and Planned Features The following are absent from the OptiFlag feature set as of 0.6.5.
  1. Ordering -- There is now way, yet, to enforce that a flag, switch or a keyword will appear in a certain order or before or after another flag, switch or keyword.
  2. Argument -- This one will appear as part of the 0.7 release and requires 'Ordering' as well. An 'argument' is a placeholder for a non-flag non-switch argument that will be fed directly into the program. It is akin to an ordered tuple in method dispatch (as opposed to keyword binding). It is probably more useful. See this discussion for more.
  3. Specification Errors -- This happens when two or more declarations conflict. One example of this would be if something like this happened:
    module CatsAreEvil extend OptiFlagSet
       character_flag :a,  :cat_group
       character_flag :c,  :cat_group
       character_flag :t,  :cat_group
    
       optional_switch_flag "cat"
    
       and_process!
    end

    Now, since the character flags cluster, (as part of their own group -- see clustering), they could cluster in any of the following ways:

    >ruby rainingCats.rb -tac
    >ruby rainingCats.rb -act
    >ruby rainingCats.rb -cat

    BUT it is this last one which conflicts with the definition of

       optional_switch_flag "cat"

    So how do we resolve this? Does this prevent the rest of the application from running? Do we have special declare-time (compile-time errors)? Not sure yet.

    This is just ONE example of a specification or declaration error. I believe the above description of these errors as compile-time is appropriate as this would be an error made by the programmer in programming in this new DSL. There could be many more compile-time errors.

  4. Commands and Sub-Commands -- See this discussion item on this missing feature. It would seem to be a very hard issue to resolve properly.
  5. Customized Help -- this actually should be possible now, as the infrastructure is all in place to register help handlers. This WILL be in release 0.7.
  6. Type Coercion -- a lot of command-line libraries provide for this. I am not so sure it's really worth it. I figure once a person has access to the string result, they can coerce it into any type they need.
  7. MUTEX Flags -- This would be an interesting feature that says if one flag is present than antoher one CANNOT be present. Not sure how hard or easy this would be to implement, or what release it could be ready for.
  8. Halt On -- This feature would allow the programmer to terminate the command-line parsing immediately if a certain flag or condition is met. This is how the help flag works (although it is handled internally). It is debatable whether we really need to offer this if we provide the feature to run an arbitrary block of code.
  9. Execute -- This is actually how most of the command-line libraries operate and why the author found them so very hard to read (mentally parse). The idea is you register a block (call it a handler or a listener) as a callback for once the flag is immediately matched. This differs from the OptiFlag philosophy where declaration and usage are separated. Nevertheless, such a functionality would allow the programmer certain flexibility.

The 80/20 Rule

The 80/20 Rule

The 80/20 rule says that 80% of the functionality of some task can be accomplished with 20% of the effort. The last 20% of the functionality takes 80% of the time/effort/sweat/tears. Idiomatically, "The devil is in the details".

I mention this because as OptiFlag grows, certain design decisions and existential questions could be posed.

1) Does OptiFlag purport to support every possible command-line scenario that exists out there?

Certainly, the usage scenarios catalog I have been mainting shows that this *might* be a never-ending task. And realizing that every single dispatch mechanism might have to be accounted for, might never make this practical or possible.

2) Do we want OptiFlag to be defeat the other command-line libraries? Or just have its own ecosystem?

An argument could be made that we stick with the 80/20 rule, implement the stuff that 80% of the people need or want to get a quick dirty command line library. This argument implicitly says: "OptiFlag is for the common people. If you want advanced command-line processing, go somewhere else." GIMP does not want to be Photoshop. A Corrolla is not a Porsche. You pay for what you get. In this case, you pay in effort to learn a new library. Remember how simple

So, putting aside the mid-life crisis issues that OptiFlag has stirred, we can still venture to answer, what exactly is the 80% of the functionality that 80% of the people (i.e. those NOT trying to implement a command like POSIX find) would need.

Essentially this is asking for what requirements/features would bring the biggest ... *still under construction*

Comparing Ease of Use

Comparing Ease of Use One of the goals of OptiFlag was to make the usage of command-line processing as easy and intuitive to use as possible. Since Ruby has excellent meta-programming facilities and excellent syntactic-sugar, it is not too difficult to embed a DSL for just such a functionality.

In the author's opinion, the "ease of use" of a command-library has two different manifestations.
  1. Declaration
  2. Usage
  3. The Extras

Declaration

How does one declare the existence of a flag/option/command-line parameter? If you have just written a script (or a full blown program), all you really want to do is get certain values into the program as quick as possible. You really just want to declare that certain values should be exposed as command-line switches. The addtional attributes about this exposition are where things get complicated. Questions arise about:
  1. Cardinality -- how many paramaters after the flag?
  2. Optionality -- is this flag required?
  3. Order -- are the flags expected in a certain order?
It is in presenting a syntax for these extra attributes that the declaration of flags becomes tricky.
*still under construction*

Usage

Now that you have declared a set of flags, the question is how to use them? *still under construction*

Category of Command Line Usage Patterns

Category of Command Line Usage Patterns The following section is my ongoing list of command-line patterns that are commonly used in command-line programs. The possibilities are legion. Any command-line parser library must make a decision: will it try to handle every single possible command-line switch/flag/parameter that

Clustering

> ls -aFl
> ls -a -F -l
> tar -xvf warez.tar
> tar -x -v -f warez.tar

A flag or a number

> tail -f /var/log/access.log
> tail -300 /var/log/access.log

Ideas for OptiFlag functionality (Flag Set Constants)

Ideas for OptiFlag functionality (Flag Set Constants) It should be possible to do something like this
PosixOptiFlagSet = OptiFlagSet(:flag-symbol => '-', :long-symbol => '--', :default-version => true)
WindowsOptiFlagSet = OptiFlagSet(:flag_symbol => '/' )
	
Which allows us to do some interesting conditional logic like this.
OurFlagSet = PosixOptiFlagSet # this is the default
OurFlagSet = WindowsOptiFlagSet if RUBY_PLATFORM =~ /(win|w)32$/
module DBChecker extend OurFlagSet
   flag "log"
   flag "password"
   flag "user"
   
   and_process!
 end 		

Ideas for OptiFlag functionality (Argument)

Ideas for OptiFlag functionality (Argument) If a user does not want to have to write the additional switch or flag to indicate and argument for his/her program,
# file helloWorld.rb
module ArgumentArgs extend OurFlagSet
   argument "username"
   argument "password"
   
   and_process!
 end

puts "     User has input #{ArgumentArgs.flags.username} for username"
puts " and user has input #{ArgumentArgs.flags.password} for password"
And the following command-line would bind "daniel" to username and "ch3ckm8" to password.
>ruby helloWorld.rb daniel ch3ckm8

The reason that this has not been implemented yet (as of 0.6.5) is that this would require ordering, which itself has not been implemented

Ideas for OptiFlag functionality (Flag clustering)

Ideas for OptiFlag functionality (Flag clustering) Ideally, flag clustering should be declaratively done like this:
module DBChecker extend OurFlagSet
   cluster do 
      character_flag :x
      character_flag :v
      character_flag :f
   end 
   cluster do 
      character_flag :l
      character_flag :s
      character_flag :a
   end 
   
   and_process!
end 		
So that 'x','v', and 'f' can cluster and 'l','s' and 'a' can independently cluster.
> ls -lsa -xvf
> ls -sa -l -x -v -f
> ls -a -ls -fv -x

Ideas for OptiFlag functionality (Sub Commands)

Ideas for OptiFlag functionality (Sub Commands)

Subcommands seem to be a powerful advanced feature of many program's command line offerings. The idea of a subcommand is most often seen in the command-line for cvs or subversion where the very first place in the command-line dispatches to a different command.

For instance, the subversion checkout subcommand 'co' might have a flag that is ONLY applicable to it (I am making up the flag -force)
> svn co -force
Whereas the subversion update command might have its own ('delete' again is an invented flag/place)
> svn clean -delete -v
Maybe, we declare something like this
module DBChecker extend OurFlagSet
   command "co" do 
      alternate "checkout"
      optional_flag "force"
   end 
   command "cl" do 
      alternate "clean","cleanup"
      optional_character_flag :v :description => "Verbose"
      optional_flag "delete"
   end 
   
   and_process!
end 		

Hmmm.... but then how do we access the data? We need to always think about that. Maybe something like:

module DBChecker extend OurFlagSet
   command "co" do 
      alternate "checkout"
      optional_switch_flag "force"
   end 
   command "cl" do 
      alternate "clean","cleanup"
      optional_character_flag :v :description => "Verbose"
      optional_switch_flag "delete"
   end 
   
   and_process!
end 	

if DBChecker.co?
   puts "You have chosen to checkout... "
   if DBChecker.co.force?
      puts "    and with force "
   end
elseif DBChecker.cl?
   # etc..
end	

Please be aware that none of the OptiFlag Ruby in this section is guaranteed to work. It's just speculation and requirements gathering and wishes at this point.