Skip to content

migrate pending data sources - #236

Open
ewollesen wants to merge 4 commits into
masterfrom
eric-migrate-pending-data-sources
Open

migrate pending data sources#236
ewollesen wants to merge 4 commits into
masterfrom
eric-migrate-pending-data-sources

Conversation

@ewollesen

@ewollesen ewollesen commented May 15, 2026

Copy link
Copy Markdown
Contributor

BACK-4414

@ewollesen

Copy link
Copy Markdown
Contributor Author

/deploy qa4

@tidebot

tidebot commented May 15, 2026

Copy link
Copy Markdown
Collaborator

ewollesen updated values.yaml file in qa4

@tidebot

tidebot commented May 15, 2026

Copy link
Copy Markdown
Collaborator

ewollesen updated flux policies file in qa4

@tidebot

tidebot commented May 15, 2026

Copy link
Copy Markdown
Collaborator

ewollesen deployed clinic eric-migrate-pending-data-sources branch to qa4 namespace

@ewollesen
ewollesen requested review from darinkrauss and toddkazakov and removed request for darinkrauss May 19, 2026 10:20
@ewollesen
ewollesen force-pushed the eric-migrate-pending-data-sources branch 2 times, most recently from 32828a8 to 5e352dd Compare May 19, 2026 11:34
darinkrauss
darinkrauss previously approved these changes May 20, 2026

@darinkrauss darinkrauss left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

// Prepend, so the most recent request is stored first
"$position": 0,
update := bson.M{
"$push": bson.M{

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you want to update the updatedTime field? That seems to have been dropped. Or are we considering the provider connection requests a "supplementary" object that doesn't necessarily imply changing the patient itself?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, any concerns with this array being unbounded? Probably not from a real-world-use perspective.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I know this hasn't changed, but we are storing the ProviderName twice, once as the key and once in the value.

@ewollesen ewollesen May 20, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or are we considering the provider connection requests a "supplementary" object that doesn't necessarily imply changing the patient itself?

This. Since there's no functional or user visible change to the patient, I'm not touching their updatedTime field.

Also, any concerns with this array being unbounded? Probably not from a real-world-use perspective.

Agreed. I have concerns, but this code predated my changes, and it's a necessary feature to see the history of connection requests.

Also, I know this hasn't changed, but we are storing the ProviderName twice, once as the key and once in the value.

Yeah, it annoys me too sometimes, but other times its proven useful to have it in both places.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ewollesen Databricks uses the patient's updated time to pull new changes. updatedTime ought to be updated when a change to the patient object is made, so those changes can be reflected in all dashboards managed in databricks.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ewollesen Databricks uses the patient's updated time to pull new changes. updatedTime ought to be updated when a change to the patient object is made, so those changes can be reflected in all dashboards managed in databricks.

Oh you're right. I was thinking of the migration when I wrote this wasn't a material change, not for when providerConnectionRequests are being added/modified. Will fix.

@ewollesen
ewollesen marked this pull request as ready for review May 26, 2026 09:59
// Prepend, so the most recent request is stored first
"$position": 0,
update := bson.M{
"$push": bson.M{

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ewollesen Databricks uses the patient's updated time to pull new changes. updatedTime ought to be updated when a change to the patient object is made, so those changes can be reflected in all dashboards managed in databricks.

@ewollesen
ewollesen force-pushed the eric-migrate-pending-data-sources branch 2 times, most recently from e442b47 to 88950ea Compare May 26, 2026 13:06
@ewollesen
ewollesen requested a review from toddkazakov May 26, 2026 13:09
@ewollesen

Copy link
Copy Markdown
Contributor Author

/deploy qa4

@tidebot

tidebot commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

ewollesen updated values.yaml file in qa4

@tidebot

tidebot commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

ewollesen updated flux policies file in qa4

@tidebot

tidebot commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

ewollesen deployed clinic eric-migrate-pending-data-sources branch to qa4 namespace

@ewollesen
ewollesen force-pushed the eric-migrate-pending-data-sources branch 3 times, most recently from 8e1f560 to 2cedef5 Compare June 17, 2026 07:36

@toddkazakov toddkazakov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Xealth we display the state of the data source (see https://github.com/tidepool-org/clinic/blob/master/xealth/program.go#L62-L67). I believe we should change that logic so we can display similar information, but this time based on the provider connection requests.

Comment thread xealth/xealth.go Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This path will still create a pending data source, even though it's not a valid value anymore

Comment thread api/mappers.go Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This block should be removed, now that data sources are marked as read only in the spec

@ewollesen
ewollesen requested a review from toddkazakov June 18, 2026 11:02
@ewollesen
ewollesen force-pushed the eric-migrate-pending-data-sources branch from 3703ff2 to 8fe3b7f Compare June 25, 2026 07:09
The expiration time of a pending provider connection belongs to the
provider connection request itself, not to a placeholder data source.
Expose it as a read-only field on the API, and have the service layer
assign both createdTime and expirationTime, as they're business logic
decisions that belong in the service layer rather than with callers.

BACK-4414
Adding a provider connection request no longer creates or updates a
placeholder dataSource with a pending or pendingReconnect state. The
provider connection request itself now carries all the information
needed to represent a pending connection, so the request is simply
prepended, and the patient's updatedTime is bumped.

BACK-4414
New xealth orders now create provider connection requests directly,
instead of placeholder dataSources with a pending state.

BACK-4414
...and mark dataSources as read-only, since they are only ever written
by the clinic-worker CDC handler. Existing placeholder dataSources are
migrated by the back-4414-migrate-pending-data-sources script in
tools-private.

BACK-4414
@ewollesen
ewollesen force-pushed the eric-migrate-pending-data-sources branch from 8fe3b7f to 26eef10 Compare July 21, 2026 08:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants