Skip to content

Rewrite statvar content to conform to new guidelines - #719

Open
kmoscoe wants to merge 42 commits into
datacommonsorg:masterfrom
kmoscoe:variables
Open

Rewrite statvar content to conform to new guidelines#719
kmoscoe wants to merge 42 commits into
datacommonsorg:masterfrom
kmoscoe:variables

Conversation

@kmoscoe

@kmoscoe kmoscoe commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

The custom entities file will come in a separate PR.

Staged at http://bullie.svl.corp.google.com:4000/custom_dc/custom_data.html

Please ignore all the formatting changes; my editor decided it was time to comply with some standard I never heard of.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the custom data documentation to use a WHO adolescent smoking dataset as an example instead of the previous adult smoking dataset, updating the MCF, CSV, and JSON configuration examples accordingly. The review feedback highlights several formatting and syntax issues in the new examples, such as broken markdown table rows, typos, an incomplete markdown link, inconsistent schema prefixes and duplicate lines in the MCF block, and incorrect delimiters in the CSV example.

Comment thread custom_dc/custom_data.md Outdated
Comment thread custom_dc/custom_data.md Outdated
Comment thread custom_dc/custom_data.md Outdated
Comment thread custom_dc/custom_data.md Outdated
Comment thread custom_dc/custom_data.md Outdated
Comment thread custom_dc/custom_data.md Outdated
Comment thread custom_dc/custom_data.md
kmoscoe and others added 4 commits July 14, 2026 13:23
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@kmoscoe
kmoscoe requested a review from carojahn July 14, 2026 23:23
Comment thread custom_dc/custom_data.md

```
Node: dcid:who/Ratio_Smokers_Age13to17
Node: dcid:who/Ratio_Smokers_Adolescents

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is who/ doing in the definition?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick but if dcid is a strict replication of the values in property it should be Ratio_Smoking_Adolescent

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WHO. The prefix is explained below.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick but if dcid is a strict replication of the values in property it should be Ratio_Smoking_Adolescent

Yeah, but I think we can add a little leeway just to make it a bit more English-y, no?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should your property value be more English-y in that case?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which property? I don't understand.

Comment thread custom_dc/custom_data.md
Comment thread custom_dc/custom_data.md Outdated
Comment thread custom_dc/custom_data.md

```
Node: dcid:who/Ratio_Smokers_Age13to17
Node: dcid:who/Ratio_Smokers_Adolescents

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick but if dcid is a strict replication of the values in property it should be Ratio_Smoking_Adolescent

Comment thread custom_dc/custom_data.md Outdated
@kmoscoe
kmoscoe requested a review from carojahn July 22, 2026 20:23
Comment thread custom_dc/custom_data.md

```
Node: dcid:who/Ratio_Smokers_Age13to17
Node: dcid:who/Ratio_Smokers_Adolescents

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should your property value be more English-y in that case?

Comment thread custom_dc/custom_data.md
Comment thread custom_dc/custom_data.md
typeOf: schema:StatisticalVariable
name: "Prevalence of current cigarette smoking among adolescents (%) [Female]"
name: "Prevalence of current cigarette smoking among adolescents (female)"
description: "Percentage of adolescents who are current cigarette smokers and are female"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not strongly opinionated about it: If we are saying that Percentage is a unit - is this really the percentage or the proportion of?

Nitpick: the description right now feels like a different word order of the name vs adding more value.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only reason I put it in is that having slightly different wording might help with embeddings generation. But maybe not. Do you have a better description text that would be more different?

Comment thread custom_dc/custom_data.md
constraintProperties: dcid:gender
gender: dcid:Female
measurementDenominator: dcid:Count_Person_13To17Years_Female
measurementDenominator: dcid:Count_Person_13To17Years

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you dividing by all adolescent? Vs by all female adolescent?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because I believe that's how they calculated it, according to their description:
Percentage of school-going adolescents of a specified age range who report smoking cigarettes either daily or occasionally, usually with reference to the 30 days prior to the survey. Numerator: Number of school going adolescents of a specified are range who report having smoked cigarettes either daily or occasionally. Denominator: Total number of school-going adolescents of this specified age range.
They don't seem to have distinguished the denominator by sex AFAICT.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IOW, it's the percentage of female smokers among all adolescents. Not the percentage of smokers among all females. (I believe.)

Comment thread custom_dc/custom_data.md
Comment thread custom_dc/custom_data.md Outdated

* `description`: A more detailed textual description of the variable.
* `description`: A more detailed textual description of the variable. Although this is optional, it is highly recommended, as the description is used to generate embeddings for natural-language search.
* `measurementQualifier`: This is used to qualify the measurement represented in all observations using the variable. It should be a member of an enumeration, e.g. `Weekly`, `Monthly`, `Annual`, which are members of the [StatAccumulationPeriodEnum](https://datacommons.org/browser/StatAccumulationPeriodEnum){: target="_blank"} type. For instance, if the `measuredProperty` is income, you can use `Annual` or `Monthly` to distinguish income over different periods. If the time interval affects the meaning of variable and and values change significantly by the time period, you should use this field keep them separate.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a bit more detail on this one? Assuming it's not defined elsewhere.

Raw notes:
"How was data treated or adjusted?"
Applies operational filters or scales.

Do’s:
Macroeconomics: Nominal, RealValue, or PurchasingPowerParity (PPP).
Time-series aggregation: Annual, Monthly, or Weekly (used when an observation is bundled into an accounting period rather than a snapshot).
Date adjustments: SeasonallyAdjusted, Rolling30YAvg.
Financial metrics: StandardizedCurrency vs. LocalCurrency.

Don’t’s:
• Temporal rule: Do not use for simple calendar dates; only include time when signaling a mathematical smoothing or accounting window.
• Constraints: Do not use it to express additional constraints.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH, I don't want to get too much into rules of usage here (we should publish another doc about that), but I can certainly summarize this briefly.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I wrote this up. PTAL

@kmoscoe
kmoscoe requested a review from carojahn July 30, 2026 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants