Component files

[tangaroa]

Some thoughts on the generalization of files with external components, such as html and Office with linked files, and imagining a standard for same at the OS or userspace level.

Component files

Consider data files composed of multiple components that different users may edit. These may be useful for:

• Programs allowing both global and local settings
• Collaborative work with strictly defined realms
• Web and file services with user-controlled realms

Examples

The core file may look like:

{
	foo: bar
	baz: import(~bob/component.dat)
	quux: import(~fred/component.dat)
}

The processed file may look like:

{
	foo: bar
	baz: {
		alpha: asdf
		beta: bsdf
	}
	quux {
		alpha: bits
		beta: breakers
	}
}

Editing

An editor, acting as root, should be able to edit the processed file with the result that edits made to the user-contributed part of the file are saved to the user's file.

For this to be possible, there must be a system having certain knowledge:

1. Know the contents of the old versions of the processed and pre-processed file.
2. Know what lines in the processed file are obtained from which user-edited files.
3. Know what syntax scopes in the processed file are obtained from which user-edited files.
4. Know when the processed file has been changed (using filesystem events)

Potential solutions

• Have all edits use a special command that handles the comparisons and magic, analagous to the vipw command for editing the unix password file.
• Have a special command to only handle the comparisons and magic, and expect the user to run it after editing the file. The command will then save the new file to a .bak file for further comparisons.
• Create a fake device for the file which wraps the edit around read and write filters. The read filter produces the processed file, while the write filter detects differences in the edited version and merges the differences into the appropriate files.

Security considerations

Components must be complete and self-contained so that their inclusion does not produce a side effect or syntax error. This requires that the including system have knowledge of how to parse the included data.

Components must not be able to change settings outside of their scope, unless they are specifically granted access to those settings.

Components may be denied the authority to change certain settings, and attempts to change these settings will be ignored.

Speed considerations

A component that has not changed may be cached. The operating system's regular filesystem cache may be the best solution.

If all components are unchanged, the final result may be cached.

In a future system, a component may not be a flat file but may be a function of variables, like exec("/bin/perl foo.pl 123 foo"). To cache the result of this function, a system must be able to identify every variable in the function and be able to tell if any of them may have changed from a previous run of the function. Variables are not only explicit $variables but anything else that can change and can have an effect on program flow, such as a resource that is read.

Alternatives

• As it is today, a program may have custom logic to search a user's home directory for a configuration file. There is no standard; this duplicates effort but allows for the development of alternatives.