Release/4.20.4

[aarthy-dk]

Features

• table groups: verify access to tables (a2e5f39)
• allow dismissing items from sorting selector (378be3b)
• test results: add action filter (1002c09)
• profiling: use native sampling (2cdc51a)

Bug Fixes

• upgrade command (f45cb80)
• sql: handle None when replacing params (dc29dd2)
• analytics: update query fetch (bc01a5e)
• sql: remove unnecessary str conversion for params (66fce8a)
• profiling: copy error when any query fails (3857d14)
• don't log full queries when upgrading system version (5a4b43e)
• profiling: sampling bugs in some flavors (3de8e8d)
• empty states in table groups and test suites pages (819c45c)
• test suites: remove unnecessary fields, disable export option when not configured (a35eb84)
• sql: incorrect parentheses in row_ct_pct test for mssql (0187c84)
• empty states in profiling runs and test runs pages (2a8bf49)
• empty state in profiling runs (f5c06be)
• empty state in project dashboard (cec1822)
• empty state in quality dashboard (cdc4e1d)
• empty state in score issues (21992e2)
• issue report: sample data not loading in score issues test reports (e92c114)
• data catalog: broken table create script (c06fbd7)
• data catalog: bugs in column history dialog (bbc45ba)
• data catalog: don't apple searchOptions when search is empty (960f67d)
• hygiene issues: error on exporting all issues (66d106a)
• test definitions: fix condition for disabling disposition buttons (9cee2ad)
• nav: validate query parameters (fdea797)
• queries: handle None results gracefully (900025a)
• profiling: error when running in background (d720d5a)
• cli: generate-tests command broken (ee8845f)
• redshift: remove unnecessary pre connection query (f8d48fb)
• data catalog: numbers are rounding unnecessarily (6649939)
• analytics: login usage event (877ede5)

Refactors

• database service and param replace for process execution (59a5c68)
• use bind parameters in app database sql for process execution (2902516)
• add sqlalchemy models for entities (9825592)
• use sqlalchemy models and bind params on all ui pages (6446a49)
• remove dead code (b6fb08d)
• Improving how pipeline parameters are obtained (e6c0675)

Miscellaneous

• Add example Aggregate Balance Test to Demo Data and Quickstart (a865c61)

[github-actions]

Coverage Report •

File	Stmts	Miss	Cover	Missing
testgen
__main__.py	319	319	0%	1–695
testgen/commands
run_execute_cat_tests.py	74	62	14%	23–45, 67–148
run_execute_tests.py	100	77	19%	34, 60–116, 120–132, 144–189
run_generate_tests.py	56	46	14%	15–94
run_get_entities.py	34	34	0%	1–83
run_launch_db_config.py	27	27	0%	1–89
run_observability_exporter.py	152	90	40%	36–67, 71–75, 79–101, 105–143, 147–191, 204–248, 281–295, 299–309, 313–319, 323–326
run_profiling_bridge.py	250	250	0%	1–474
run_quick_start.py	83	83	0%	1–159
run_refresh_data_chars.py	28	20	25%	19–75
run_rollup_scores.py	23	16	26%	11–20, 24–33
run_test_parameter_validation.py	73	65	8%	22–118
run_upgrade_db_config.py	82	82	0%	1–149
testgen/commands/queries
execute_cat_tests_query.py	69	30	53%	35–43, 46–49, 52–80, 84, 88, 92, 96, 100, 104, 108, 112, 116, 120, 124
execute_tests_query.py	91	38	52%	55–60, 63–92, 97–137, 141–144, 148, 152, 156–166
generate_tests_query.py	50	19	57%	32–34, 37, 53–58, 62, 66, 70–74, 78, 82
profiling_query.py	194	110	35%	66–>79, 82–85, 88, 127–138, 142, 146, 150, 154, 158, 162, 166, 170, 174, 178, 182, 186, 190–210, 214–222, 226, 230, 234, 238, 246–340, 344, 348, 352, 356, 360
refresh_data_chars_query.py	63	7	88%	55, 74, 88–93, 97, 101
rollup_scores_query.py	22	10	55%	12–13, 16–22, 26, 30, 34, 38
test_parameter_validation_query.py	39	17	54%	20–22, 25–39, 43–46, 50, 54, 58, 62, 66, 70
testgen/common
get_pipeline_parms.py	47	12	66%	45–51, 55–61, 65–71
mixpanel_service.py	71	37	44%	25–29, 39–44, 47, 51–53, 57–72, 75–78, 81–91, 95–103
testgen/common/database
database_service.py	249	193	19%	51–65, 69–70, 75, 79–83, 98–128, 138–164, 173–209, 215–224, 230–237, 241–261, 273–285, 295–297, 306–348, 352–382, 387–389, 392–416
testgen/common/database/flavor
databricks_flavor_service.py	7	7	0%	1–12
flavor_service.py	75	33	51%	42–67, 70, 73, 76, 79–84, 88, 92
mssql_flavor_service.py	20	20	0%	1–34
redshift_flavor_service.py	10	10	0%	1–17
snowflake_flavor_service.py	33	33	0%	1–62
trino_flavor_service.py	9	9	0%	1–15
testgen/common/models
connection.py	95	32	59%	69–70, 75–79, 86–87, 91–94, 98–99, 103–106, 110–113, 116–126
custom_types.py	37	11	63%	14, 22–24, 27–29, 37, 45, 53, 56
entity.py	112	62	40%	25, 28–31, 43–48, 52, 62–76, 81–83, 87, 98–109, 113, 117–122, 126, 130, 134–135, 139, 142–152, 155–159, 162–165
profiling_run.py	131	131	0%	1–259
project.py	50	12	73%	42–84, 88–89, 93–96, 100–101
scheduler.py	31	6	81%	28–29, 33, 36–37, 41
scores.py	296	158	39%	108–120, 124–131, 140–149, 152–156, 159–162, 174–213, 228–258, 274–315, 331–358, 368–390, 393–399, 405, 460, 466–478, 481–482, 492, 496–511, 541–545, 548–549, 600–608, 611–615, 647–654
table_group.py	141	48	61%	99–100, 107–108, 113–182, 186–203, 207–214, 218–254, 258–261, 264–279
test_definition.py	230	51	74%	107–111, 218–227, 234–241, 248–255, 264–282, 293–318, 329–360, 364–365, 368–381
test_run.py	126	39	66%	98–103, 107–117, 128–209, 213–215, 219–223, 227–231, 235–248, 252–254, 257
test_suite.py	104	28	72%	74–75, 82–83, 88–181, 185–193, 197–204, 208–239, 243–246
testgen/ui
app.py	39	39	0%	1–77
session.py	57	16	65%	52, 55, 58–60, 63–65, 69–79
testgen/ui/components/widgets
sidebar.py	27	12	48%	40, 64–80
testgen/ui/navigation
page.py	47	26	36%	25–30, 33–49, 52–59, 63
router.py	76	56	22%	25–26, 29–67, 70, 77–79, 82, 85–113, 116–119, 122–128
testgen/ui/pdf
hygiene_issue_report.py	53	53	0%	1–203
test_result_report.py	73	53	22%	47–163, 167–199, 203–218, 222–226, 230–252, 259–260
testgen/ui/queries
profiling_queries.py	71	49	26%	73–119, 130–135, 145–150, 160–165, 176–259, 270–275, 288–298, 308–313, 323–328, 339–416, 421–470
scoring_queries.py	38	38	0%	1–212
source_data_queries.py	104	79	19%	24–87, 95–150, 158–182, 196–217, 223–232
table_group_queries.py	56	56	0%	1–100
test_result_queries.py	20	12	40%	19–121, 126–162
testgen/ui/services
database_service.py	43	28	35%	23–30, 34–36, 42–46, 50–53, 57–65
user_session_service.py	64	41	31%	21–28, 34–45, 49–58, 62–68, 72–83, 94, 98, 102, 106, 110
testgen/ui/views
connections.py	225	225	0%	1–407
data_catalog.py	171	171	0%	1–548
hygiene_issues.py	216	216	0%	1–658
profiling_results.py	96	96	0%	1–308
profiling_runs.py	118	118	0%	1–252
project_dashboard.py	25	25	0%	1–45
project_settings.py	76	76	0%	1–145
quality_dashboard.py	21	21	0%	1–52
score_details.py	99	99	0%	1–216
score_explorer.py	170	170	0%	1–373
table_groups.py	177	177	0%	1–347
test_definitions.py	521	456	10%	49–180, 186–208, 212–216, 234–634, 640, 646, 657–751, 754–763, 768–786, 792–812, 822–826, 838–965, 974–1001, 1011–1035, 1046–1095, 1100–1104, 1113–1142, 1150–1158, 1162–1176, 1180–1195
test_results.py	303	243	16%	64–238, 246–257, 273–274, 279–286, 291–300, 305–348, 358–387, 435–597, 608–627, 637–711, 715–725, 731–760, 764–767, 771–779, 786–829
test_runs.py	128	84	28%	48–126, 150, 153, 156–165, 169–215, 219–223, 229–275
test_suites.py	143	102	25%	44–54, 84, 90, 96–97, 101–215, 221–254, 259–295
testgen/ui/views/dialogs
generate_tests_dialog.py	68	54	16%	17–83, 87–106, 110–117, 121
manage_schedules.py	100	75	22%	25, 28, 31, 34, 38–42, 45–87, 90–163
profiling_results_dialog.py	16	6	45%	12–18, 24–27
run_tests_dialog.py	50	37	20%	19–88
testgen/utils
__init__.py	118	92	15%	23–25, 32–41, 45–53, 57–60, 65–66, 70–80, 84–100, 104–109, 113–115, 119–150, 176, 189–192, 202–215, 219–231
TOTAL	9478	6735	25%

Tests	Skipped	Failures	Errors	Time
37	0 💤	0 ❌	0 🔥	9.940s ⏱️