Retiring is_rcpch_audit_team_member and is_rcpch_staff¶
This is the plan for replacing the two Epilepsy12User boolean flags with permissions attached to the RCPCH Audit Team role. Code checks those permissions with has_perm, not group membership.
Status: Phase 3a done; Phase 3b (archive and drop the columns) still to do. National access now comes only from the RCPCH Audit Team role's group permissions, and no code uses the flags. The database columns stay until Phase 3b.
How the code works now¶
- Two separate systems. The
RCPCH_AUDIT_TEAMrole (4) puts a user in theepilepsy12_audit_team_full_accessgroup (common_view_functions/group_for_group.py). The two booleans are set on their own, and about 70 non-test call sites check them directly. - They can disagree. The model's comments say audit team members "can be clinicians". So a Lead Clinician with
is_rcpch_audit_team_member=Truegets national access without the role or group. This is the main migration risk (see Phase 0). - Permissions already exist for this.
groups_seeder()inmanagement/commands/create_groups.pygives the audit team group its ownEPILEPSY12_AUDIT_TEAM_ACCESS_PERMISSIONSlist. That's where the new permissions go. - Superuser checks get simpler.
has_perm()is always True for an active superuser, so everyis_superuser or flagbecomes a singlehas_perm(...).
New permissions¶
Define each new permission in constants/user_types.py, add it to PERMISSIONS and to the Meta.permissions of the model it most relates to, and attach it only to the audit team group. Where a built-in model permission already fits (view_banner), use that instead.
| Permission | Replaces the flag checks in |
|---|---|
can_access_all_organisations (view any organisation, user or child nationally, including the national view preference and the submission dashboard) |
decorator._user_may_view_this_organisation, user_may_view_organisational_audit, user_can_access_user, lookup_user_permissions_on_child (can_view), permissions.CanAccessOrganisation, organisation_views.py:120,381, case_views.py:1164, user_management_views.py:149,209,495, sanction_user_access.merged_parent_list_for_user (Jersey rule), audit_period_permissions._is_rcpch_user (5 callers, all view checks), case_table.html:117; national view: Epilepsy12User.save() invariant, organisation_views.py:938 toggle, rcpch_choices in user_management_views.py:332 and case_views.py:455, partials/cases/view_preference.html; admin data banner and submission dashboard link: nav.html:5,27, and the dashboard view itself (decorator.rcpch_full_access_only) |
can_edit_case_outside_audit_period (edit any child whatever its organisation, lock state or period) |
lookup_user_permissions_on_child (can_edit), registration_views.py:612 (earliest date = DOB), registration_dates.html:18 (change a registration date), case_table.html:136–150 |
can_manage_rcpch_audit_team_users (create/edit RCPCH audit team users, edit emails) |
decorator.py:74 (user_type == "rcpch-staff"), Epilepsy12UserAdminCreationForm.__init__ / clean(), epilepsy12_user_list.html:31, admin_create_user.html:73 |
view_banner (built-in; see banners targeted at any role, to check how they look) |
context_processors/banner.py:19 |
can_approve_audit_period_organisations |
approve_audit_period_organisations.py:117 |
Line numbers are as of the commit this plan was written against.
No new permission needed for two of them:
- KPI download and publishing:
kpi_downloadandkpi_download_filealready sit behindcan_publish_epilepsy12_data, so the flag check inside them is redundant and can go.cohort_card.html:56and the publish button inselected_organisation_summary.html:30useperms.epilepsy12.can_publish_epilepsy12_data. - 16-character admin passwords:
Epilepsy12UserUpdatePasswordFormchecksrole == RCPCH_AUDIT_TEAM or is_superuser, nothas_perm. The form also runs for password resets and first-time setup, when the account may still be inactive, andhas_permis always False for inactive users.
Where code needs to know what a user is, not what they may do, use user.role == RCPCH_AUDIT_TEAM rather than a group or permission check:
create_userneeds no organisation for this role (groups aren't assigned yet at that point).- The "Edit RCPCH Epilepsy12 staff member" title (
user_management_views.py:594). - The user table badges.
Phases¶
Phase 0 – data audit (before any code change)¶
Run the read-only report (removed in Phase 3, once the flags were gone):
python manage.py audit_rcpch_user_flags
python manage.py audit_rcpch_user_flags --active-only
python manage.py audit_rcpch_user_flags --csv --output rcpch_flags_report.csv
It lists users with any of these findings:
flag_without_role: a flag is set but the role is not RCPCH Audit Team. These users lose national access after the switch.role_without_flag: the role is RCPCH Audit Team but a flag is False. These users gain whatever that flag gates.group_mismatch: the user's groups are not exactlygroup_for_role(role).
The team decides each one: change the role, or accept the change in access. Without this step, some users will lose access without warning.
The test seed users *_CLINICAL_AUDIT_TEAM@nhs.net (tests/UserDataClasses.py) are Lead Clinicians with both flags set, so they show up as flag_without_role. Revisit them in Phase 4.
Phase 1 – add the permissions (additive, safe to deploy)¶
-
Add the constants and
Meta.permissionsentries, then runmakemigrations(permission-only migration,0074_rcpch_audit_team_permissions). Each permission sits on the model it most relates to:Permission Model can_access_all_organisationsOrganisationcan_edit_case_outside_audit_periodCasecan_manage_rcpch_audit_team_usersEpilepsy12Usercan_approve_audit_period_organisationsAuditPeriodOrganisation -
Extend
EPILEPSY12_AUDIT_TEAM_ACCESS_PERMISSIONSingroups_seederwith the four new permissions and the built-inview_banner, which no group held before. - Extend
tests/general_functions_tests/test_groups_seeder.py: the audit team group has all five (includingview_banner), the other groups have none.
Deploying. migrate creates the permission rows; it doesn't attach them to the group. s/start-prod runs the seeder on every start, straight after migrate. Any environment that starts another way needs it run by hand:
python manage.py seed --mode=seed_groups_and_permissions
Until Phase 2, nothing checks these permissions, so the gap between deploying and running the seeder doesn't affect anyone.
Groups. The re-sync data migration originally planned here was dropped. rcpch/rcpch-audit-engine#1496 lets audit team members and superusers change a user's role, which re-syncs that user's groups. Re-run audit_rcpch_user_flags and fix any remaining group_mismatch by setting the role again.
Phase 2 – switch the checks¶
- Replace each call site in the table with
user.has_perm(PERM_...)(full names are constants inconstants/user_types.py), orperms.epilepsy12.<codename>in templates. audit_period_permissions._is_rcpch_usercheckscan_access_all_organisations. All five callers are view checks; editing stays constrained byCase.editable().Epilepsy12User.save()checksrole == RCPCH_AUDIT_TEAM or is_superuserfor the national-view invariant. Django caches permissions on the user object, and a role change only updates groups after the save, sohas_permwould be wrong there.decorator.rcpch_full_access_only(the submission dashboard) checkscan_access_all_organisationsinstead of group membership, matching the nav link. Superusers now get access too.- A database system check (
epilepsy12/checks.py,epilepsy12.E001) errors ifepilepsy12_audit_team_full_accesslacks any permission inRCPCH_AUDIT_TEAM_REQUIRED_PERMISSIONS, so a missed seed shows up instead of locking the audit team out. Django runs database checks onmigrateandmanage.py check --database default, not on server start. The check skips a fresh database (no group yet) and permission rows that don't exist yet, becausemigrateruns it before applying migrations. - Tests: the seeded
CLINICAL_AUDIT_TEAMuser now has the RCPCH Audit Team role, and test helpers that build users add them to their role's group, as real users are.
Phase 3 – remove the fields¶
Deploys (s/ci) start a new Azure Container Apps revision, which runs migrate (s/start-prod) while the previous revision is still serving. Dropping the columns in the same release that stops using them would break every request that loads a user until the previous revision is gone. It could also miss history rows the previous revision writes during the archive copy. So Phase 3 is two releases (expand/contract).
Phase 3a – stop using the fields.
- Migration
0075_remove_rcpch_user_flags_from_modelsfirst writes aVisitActivityentry (activity 13, "Superuser or admin status changed") for every user with either flag set, e.g.RCPCH flags at removal: is_rcpch_audit_team_member=True, is_rcpch_staff=False. That records who held them at removal, including users created with a flag set, who have no earlier record. It then gives both columns a database default ofFalse, on the user and history tables. Then it removes the fields from Django's model state only (SeparateDatabaseAndState). The columns stay: the previous revision keeps reading and writing them, and inserts from this release onwards get the default. create_userrequires an organisation unlessrole == RCPCH_AUDIT_TEAM(previously: unlessis_rcpch_staff).REQUIRED_FIELDSandcreate_superuserno longer mention the flags.- Removed the flags from:
- the user form's
save(), and the admin's list display, fieldsets andsave_model, which used to sync them from role. The admin still re-syncs groups on a role change. Epilepsy12UserSerializerand the user list CSV export;signals.py.roleis now inEMAIL_TRIGGER_FIELDS: promotion to the RCPCH Audit Team used to email admins only because the flags changed with it.create_dev_users,user_scriptsandmanual-setup.md.
- the user form's
- The user table's two flag badges are now one "RCPCH Audit Team Member" badge, shown by the display-only template filter
has_rcpch_audit_team_role. - Deleted the
audit_rcpch_user_flagscommand and its tests.
Phase 3b – archive the history and drop the columns¶
Ship this only once Phase 3a is live everywhere, so no running revision uses the columns.
- Create
RetiredUserFlagHistory, a read-only archive (viewable in the Django admin). Copy the flag values from everyHistoricalEpilepsy12Userrow into it, so the record of who held the flags, and when, stays in the platform. Then drop the four columns. - The fields are no longer in Django's model state, so the copy and the drop use SQL (
RunPythonwith a cursor, andRunSQLwith a reverse that re-adds the columns). A first version, using the ORM, is in the history of rcpch/rcpch-audit-engine#1504; adapt it. - Rows saved after Phase 3a will be archived with
False, which is accurate: nobody holds the retired flags then. - Flag changes are also still in
VisitActivity.detailsas text.
Phase 4 – tests¶
Done alongside Phases 2 and 3:
- The factories and fixtures (
E12UserFactory,seed_users,UserDataClasses) no longer set flags. The seededCLINICAL_AUDIT_TEAMuser has the RCPCH Audit Team role, and test helpers add users to their role's group. - Tests that asserted flag values now assert role and group, and lookups by flag now filter on
role=RCPCH_AUDIT_TEAM. test_groups_seeder.pychecks the audit team group holds every RCPCH Audit Team permission and the other groups hold none.
Existing bugs fixed along the way¶
- Missing app label (Phase 2):
add_employer_organisationcalledhas_perm("can_allocate_user_to_organisation")without theepilepsy12.prefix, so it was always False for non-superusers. - Typo:
epilepsy12_user_list.htmlcheckeduser.is_rcpch_team_member, which doesn't exist. It now checkscan_manage_rcpch_audit_team_users.
Behaviour changes to sign off¶
- Inactive superusers: they lose implicit access, because
has_permreturns False for inactive users. The current flag checks don't look atis_active. - Clinicians with the flag: a clinician who had
is_rcpch_audit_team_member=Truebut a trust role loses national access unless their role changes in Phase 0.
Suggested PRs¶
- Phases 0–1.
- Phase 2.
- Phases 3a and 4, after Phase 2 has run in production.
- Phase 3b, after Phase 3a is live everywhere.