Get Access
We are currently in private access.
Meet with us virtually to:
Request access
See a live demo
Ask us anything
We are currently in private access.
Meet with us virtually to:
ULTRA-metrics is a framework that measures experience, skills, and autonomy for any ultrasound user and application.
Delphi-Derived with diverse ultrasound experts globally
Integrates commonly used scoring metrics
* Peer-review in progress
To grow ultrasound capacity by addressing four areas of need.
Competency guidelines vary widely and overly rely on exam counts rather than direct measures of skill or autonomy.
Expert shortages limit ultrasound training capacity, yet learners are typically held to fixed supervision and exam requirements regardless of how quickly they progress.
Promising tools like virtual mentorship and AI could expand training capacity, but slow validation hinders their adoption.
Inconsistent competency metrics across programs make it difficult to benchmark performance or conduct meaningful multicenter research.
Designed for any ultrasound program - focused on four design principles.
Measure competency holistically (beyond exam counts) with additional metrics to quantify experience, autonomy, and skills.
Ensure wide applicability across ultrasound applications, regions, and training programs by using simple, robust, and generalizable measurements.
Facilitate comparison of datasets through development of framework common data elements (CDEs).
Ensure the framework is modular so it can evolve over time as new ultrasound competency metrics and training methods become available.
Enables programs to use only what they need - allows the framework to evolve where it needs to over time.
Foundational pillars that capture every dimension of competency.
(e.g. experience, autonomy, skills)
Domain-specific measures that convert performance into trackable, comparable data.
(e.g. interpretation accuracy)
Flexible scoring scales (binary, 3-point, 5-point) to allow programs choose their desired level of measurement granularity.
(e.g. entrustability scored with 3 or 5 point scale)
Clear criteria for every score to boost data quality and inter-rater consistency.
(e.g. entrustability scored with 3 or 5 point scale)
Contextual ultrasound exam info to improve data analysis and cross-program comparisons.
(e.g. exam type, cohort, clinical vs. training)
Micro-skill checklists that reveal exactly where learners excel or struggle to enable precise, data-driven feedback.
(e.g. acquisition depth, gain, preset, etc.)
Only 3 metrics and 6 metadata items are required start - optional add-ons available as needed.
Explore core metrics below - the toolkit allows you to download the full list of metrics including metadata and skill trees.
Competency Domain
Quantity of ultrasound exams performed and diversity of clinical scenarios.
Metric
Number of exams performed of each type (i.e. Cardiac, Lung, etc.)
Metric (optional)
Number of findings interpreted by exam type (i.e. Lung B-lines)
Competency Domain (optional)
Degree of independence performing ultrasound.
Metric (optional)
Trust in a learner's readiness to perform certain clinical tasks independently based on demonstrated competence.
Answer Set
Score | CLassifier | Criteria |
|---|---|---|
1 | Dependent | Supervisor did it: trainee required complete guidance or was unprepared; supervisor had to do most of the work |
2 | Guided | Supervisor talked through it: trainee was able to perform some tasks but required repeated directions |
3 | Prompted | Supervisor needed to prompt: trainee demonstrated some independence and only required intermittent prompting |
4 | Monitored | Supervisor needed to be there just in case: trainee functioned fairly independently and only needed assistance with nuances or complex situations |
5 | Independent | Supervisor did not need to be there |
Competency Domain
Level of skills performing ultrasound.
Measured across specific skill-domains.
Skill Domain (optional)
Assessing if the ultrasound exam is appropriate based on the patient’s history, symptoms, and other findings, with a clear clinical question to guide its use.
Metric (optional)
Can the practitioner articulate a focused clinical question that ultrasound can address for this patient?
Answer Set
Score | CLassifier | Description |
|---|---|---|
1 | Poor | No relevant clinical question formulated |
3 | Adequate | Relevant clinical question formulated, but it lacks specificity to the patient's condition |
5 | Ideal | Minimal criteria met for diagnosis, all structures imaged with excellent image quality |
Skill Domain
Acquiring and optimizing ultrasound images.
Metric
Were the acquired ultrasound scans of enough completeness and quality to answer the clinical question?
Two options available to capture this metric.
Option 1
Uses a common scoring method from North America; however may not account for scenarios of high quality but low completeness and vise-versa.
Answer Set
Score | CLassifier | Description |
|---|---|---|
1 | Poor | No recognizable structures |
2 | Fair | Minimally recognizable structures but insufficient for diagnosis |
3 | Adequate | Minimal criteria met for diagnosis, recognizable structures but with some technical or other flaws |
4 | Good | Minimal criteria met for diagnosis, all structures imaged well |
5 | Ideal | Minimal criteria met for diagnosis, all structures imaged with excellent image quality |
Option 2
Enables independent measurement of quality and completeness.
Answer Set
Score | CLassifier | Description |
|---|---|---|
1 | Poor | One or more required scan views missing |
3 | Adequate | Required scan views are complete, one or more optional scan views missing |
5 | Ideal | Required and optional scan views are complete |
Answer Set
Score | CLassifier | Description |
|---|---|---|
1 | Poor | Scans do not allow for accurate interpretation |
3 | Adequate | Scans allow for accurate interpretation, but scan quality could be improved |
5 | Ideal | Scans allow for accurate interpretation with excellent diagnostic scan quality |
Skill Domain
Distinguishing normal from abnormal findings.
Metric
Were relevant findings and pathology accurately interpreted?
Answer Set
Score | CLassifier | Description |
|---|---|---|
1 | Poor | Major inaccuracies |
3 | Adequate | Minor inaccuracies only |
5 | Ideal | No minor or major inaccuracies |
Skill Domain (optional)
Integrating ultrasound findings into clinical management.
Metric (optional)
Were ultrasound findings appropriately integrated into clinical decision-making?
Answer Set
Score | CLassifier | Description |
|---|---|---|
1 | Poor | Fails to appropriately apply ultrasound findings toclinical decisions |
3 | Adequate | Applies ultrasound findings appropriately butmay overlook subtle details or additional relevantinformation |
5 | Ideal | Appropriately incorporates all ultrasound findingsinto clinical decisions without errors or omissions |
Three tools to make implementation a breeze - compatable with almost any assessment workflow
Full list of metrics, answer sets, score criteria, metadata, skill trees, and which ones are required vs. optional.
Capture, monitor, and analyze ULTRA-metrics with a web-based software platform.
Real-time, virtual async, and retrospective data capture workflows with data privacy guidance.
Presuna software has everything you need
Mobile and web-based software for simple integration with your assessment workflows.
Cohort and user based analytics with longitduinal and snapshot views.
Comply with GDPR, HIPAA, and any other data privacy regulation.
Use alongside your scan archiving software, or use Presuna as your scan archive.
Collected by ultrasound programs and researchers
Quantitative survey responses from trainees, reviewers, and administrators on the usability and value of ULTRA-metrics in various contexts.
Anonymized longitudinal cohort data mapping exam counts to ULTRA-metrics scores across various cohorts.
Competency curves quantifying the impact of training innovations like virtual mentorship and artificial intelligence.
The Data Registry aims to improve competency guidelines, share best practices across ultrasound programs, and accelerate training innovations.
We're actively seeking contributors to grow the Data Registry.
Once it reaches a useable initial size, we'll grant public and private access to it.
To learn more or contribute, click "Get Access" below.
