refactor(phase): merge KS2+KS4 into fact_performance, fix all phase inconsistencies
All checks were successful
Build and Push Docker Images / Build Backend (FastAPI) (push) Successful in 50s
Build and Push Docker Images / Build Frontend (Next.js) (push) Successful in 1m12s
Build and Push Docker Images / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 1m24s
Build and Push Docker Images / Trigger Portainer Update (push) Successful in 0s

Root cause: the UNION ALL query in data_loader.py produced two rows per
all-through school per year (one KS2, one KS4), with drop_duplicates()
silently discarding the KS4 row. Fixes:

- New dbt mart `fact_performance`: FULL OUTER JOIN of fact_ks2_performance
  and fact_ks4_performance on (urn, year). One row per school per year.
  All-through schools have both KS2 and KS4 columns populated.
- data_loader.py: replace 175-line UNION ALL with a simple JOIN to
  fact_performance. No more duplicate rows or drop_duplicates needed.
- sync_typesense.py: single LATERAL JOIN to fact_performance instead of
  two separate KS2/KS4 joins.
- app.py: remove drop_duplicates (no longer needed); add PHASE_GROUPS
  constant so all-through/middle schools appear in primary and secondary
  filter results (were previously invisible to both); scope result_filters
  gender/admissions_policies to secondary schools only.
- HomeView.tsx: isSecondaryView is now majority-based (not "any secondary")
  and isMixedView shows both sort option sets for mixed result sets.
- school/[slug]/page.tsx: all-through schools route to SchoolDetailView
  (renders both SATs + GCSE sections) instead of SecondarySchoolDetailView
  (KS4-only). Dedicated SEO metadata for all-through schools.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-03-30 14:07:30 +01:00
parent 695a571c1f
commit 6e5249aa1e
7 changed files with 227 additions and 216 deletions

View File

@@ -45,8 +45,11 @@ export function HomeView({ initialSchools, filters, totalSchools }: HomeViewProp
const isLocationSearch = !!searchParams.get('postcode');
const isSearchActive = !!(hasSearch || searchParams.get('local_authority') || searchParams.get('school_type'));
const currentPhase = searchParams.get('phase') || '';
const hasSecondaryResults = allSchools.some(s => s.attainment_8_score != null);
const isSecondaryView = currentPhase.toLowerCase().includes('secondary') || hasSecondaryResults;
const secondaryCount = allSchools.filter(s => s.attainment_8_score != null).length;
const primaryCount = allSchools.filter(s => s.rwm_expected_pct != null).length;
const isSecondaryView = currentPhase.toLowerCase().includes('secondary')
|| (!currentPhase && secondaryCount > primaryCount);
const isMixedView = primaryCount > 0 && secondaryCount > 0 && !currentPhase;
// Reset pagination state when search params change
useEffect(() => {
@@ -79,13 +82,13 @@ export function HomeView({ initialSchools, filters, totalSchools }: HomeViewProp
.finally(() => setIsLoadingMap(false));
}, [resultsView, searchParams]);
// Fetch LA averages when secondary schools are visible
// Fetch LA averages when secondary or mixed schools are visible
useEffect(() => {
if (!isSecondaryView) return;
if (!isSecondaryView && !isMixedView) return;
fetchLAaverages({ cache: 'force-cache' })
.then(data => setLaAverages(data.secondary.attainment_8_by_la))
.catch(() => {});
}, [isSecondaryView]);
}, [isSecondaryView, isMixedView]);
const handleLoadMore = async () => {
if (isLoadingMore || !hasMore) return;
@@ -209,10 +212,10 @@ export function HomeView({ initialSchools, filters, totalSchools }: HomeViewProp
className={styles.sortSelect}
>
<option value="default">Sort: Relevance</option>
{!isSecondaryView && <option value="rwm_desc">Highest R, W &amp; M %</option>}
{!isSecondaryView && <option value="rwm_asc">Lowest R, W &amp; M %</option>}
{isSecondaryView && <option value="att8_desc">Highest Attainment 8</option>}
{isSecondaryView && <option value="att8_asc">Lowest Attainment 8</option>}
{(!isSecondaryView || isMixedView) && <option value="rwm_desc">Highest R, W &amp; M %</option>}
{(!isSecondaryView || isMixedView) && <option value="rwm_asc">Lowest R, W &amp; M %</option>}
{(isSecondaryView || isMixedView) && <option value="att8_desc">Highest Attainment 8</option>}
{(isSecondaryView || isMixedView) && <option value="att8_asc">Lowest Attainment 8</option>}
{isLocationSearch && <option value="distance">Nearest first</option>}
<option value="name_asc">Name AZ</option>
</select>