PreSearch

A play on words, meaning pre-research. Truly just my personal playroom to improve my skills and have fun.

AI Legality

The only legal uses of AI in this repo are:

1. Research
2. Providing examples
3. Finding and understanding documentation
4. Ensuring that important code is correctly functional before execution (.gitattributes, .gitignore, webscraping legality, etc.)
5. Audits of finished projects
6. AI self-management

All uses of AI must be documented in AI_registry.md, where additional AI usage information can be found.

Scope

This repo is intended to be wide-scoped, so each subfolder will have its own .toml.

Data Conventions

Acceptable file types include, but are not necessarily limited to:

• .csv
• .parquet
• .png
• .pdf
• .html
• .json

Raw data bares no conventions.

Cleaned

Cleaned data should be true to its name. Only directly useful information should be kept, with a general preference on reducing size in order to keep computation efficiency low down the line.

Processing

Any data that doesnt fit cleanly into raw, cleaned, or out (intermittent stages if ever necessary). In the general case, any finished project should not pass data through this folder, instead, just process stright from cleaned to output.

Out

This is the only datatype not confined to the \data\ directory. In fact, it is the convention that projects should contain their own outputs in an *\out\ subfolder within the project directory. True to it's name, the only files that should be stored here are output-ready. These files should be presentable, not just functional.

Private

Similair to \out\ in every way, private files are out files never to be published for the public eye. This might include personal financial statements or outputs from systems trained on non-commercializable inputs.

Project Convnetions

Despite having no end user, we will follow the conventions of end user-oriented technical communication:

• Code should prioritize readability over cleverness, unless the code in question is sufficiently clever
• In line with the above sanction, code should be well-commented, and variables should be sufficiently explained and meaningfully named
• All relevant outputs should be self-contained within the project, and should be properly formatted according to their potential end user
• Projects themselves, as well as any subprojects of sufficient size, should have their own dedicated READMEs

Functional files within the project directories should be numbered in sequence of execution.