PennAIR Project

This repository contains my solutions for Parts 1–5 of the PennAIR assignment.
Since videos and images are too large for GitHub, all the outputs are linked via Google Drive.

---

Part 1: Static Image Detection

Part 1 — Static Image Detection

Approach: I began by smoothing the input image using a mean-shift filtering so I could reduce the background texture and keeping strong edges. Then I applied a Canny edge detection, and then a dilation and morphological closing to make contours solid. I skipped smaller countours and the ones on the border. For all the valid contour, I approximated the polygon, drew its outline, and compute the center (using the image moments, with a circle fallback). I sorted centers row/column order and then annotated with labels on my output image.

Run command:

python3 part1_static.py --infile PennAir2024.png --out part1Solution.png

For this you would need to import the initial image and name it PennAir2024.png

Additional:

This default works fine.

If you want to tune it, you can add:

--min_area (ignore tiny blobs)

--border (ignore shapes touching edges)

--centers_json <file.json> (save centers to a JSON file)

Drive Link to Image: https://drive.google.com/file/d/1zajuqQ_6IUDYhJq3JFReVqmMPoyfnTWs/view?usp=sharing

Part 2:

Approach: I reused my Part-1 method but ran it on each and every frame like a stream (no looking ahead), writing an annotated video. I made each frame smoothed, extracted edges, closed the gaps, filtered contours, and computed/painted centers. To keep it fast, I trimmed morphology, tuned Canny thresholds, and downscaled frames if it was needed. It tracks solidly across the clip; the only slight issue is the green shape on green-ish grass background. It's mostly accurate but can jitter a bit from texture/contrast—lowering min_area or slightly increasing blur helps.

Run Command:

python3 part2_dynamic.py --infile PennAir2024Dynamic.mp4 --out part2Solution.mp4

For this you would need to import the initial video and name it PennAir2024Dynamic.mp4

Additional:

If you want to tune it, you can add:

--scale <0–1> (downscale frames to speed up)

--preview (show live window)

--print-centers (log centers per frame)

Link to Vid - https://drive.google.com/file/d/1vCctrn8XhwaxCb7UeDbYDQiXhSlEQomt/view?usp=sharing

Part 3:

Approach:

This version is both edge-first and background-aware. I estimated the background color stats by using the frame rim in LAB, z-score pixels, and the mask regions that had deviated from the background. I also computed a low-texture mask (Laplacian energy) so the busier looking asphalt gets suppressed and the smoother shapes can pass through. The two masks get combined, then cleaned with light morphology before contouring. For contours that had a lot of internal edges, I used a tiny watershed split to avoid acidentally merging adjacent shapes. This makes the detections a lot less sensitive to the lighting and background clutter.

Run Command:

python3 part3.py --infile PennAir2024DynamicHard.mp4 --out part3Solution.mp4

Note you would need to import PennAir2024DynamicHard.mp4

If you want to tune, you can use:

--scale <0–1> (downscale for speed)

--preview (show live window)

--print-centers (log centers per frame)

Link to vid - https://drive.google.com/file/d/1VobvjdpKIVu3JNEz1XAxocBxVDI267MJ/view?usp=sharing

Part 4:

Approach:

Same core detecting as Part 3. For each contour I grabbed the min enclosing circle radius r_px, and estimated a depth with a pinhole model Z = f_eff * R / r_px (known real radius R, effective focal f_eff). I then back-project to 3D X,Y,Z. This does assumes roughly front-facing disks, intrinsics close, and negligible distortion. There are some small jitters which happens on tiny/oblique shapes. To stabilise, downscale less or raise min area.

Run Command:

python3 part4.py --infile PennAir2024DynamicHard.mp4 --out part4Solution.mp4

Again, you would need to import the video.

If you want to tune it, you can add:

--scale <0–1> (speed vs. quality)

--preview (show live window)

--print-centers (log per-frame pixels + XYZ)

--fx --fy --cx --cy (camera intrinsics)

--radius_in (real circle radius for depth)

Link - https://drive.google.com/file/d/181BikG21jusrYG17nkl_Qna2egvDi4MA/view?usp=sharing

Part 5:

video_node: streams video as /camera/image_raw + /camera/camera_info

detect_node: runs the detector on each frame, publishes:

/camera/detections_image (overlay frames)

/camera/detections (Float32MultiArray: [x1,y1, x2,y2, ...])

The package: pennair_detect_ros Launch file: launch/pennair_launch.py (starts both nodes)

Run Instructions

1. Put the data where the launch expects it

Inside your project folder (host):

PennAIR/ ├─ pennair_detect_ros/ ├─ data/ │ └─ PennAirDynamicHard.mp4 # <— put your video here

2. Start a ROS 2 Humble Docker shell (inside project root) docker run -it --rm --name pennair-ros
   -v "$PWD:/ws" -w /ws
   ros:humble bash

3. Build the ROS 2 package source /opt/ros/humble/setup.bash colcon build --symlink-install source install/setup.bash

4. Launch both nodes ros2 launch pennair_detect_ros pennair_launch.py

5. Verify it’s running (new terminal in the same container) source /opt/ros/humble/setup.bash source install/setup.bash

peek one message from each:

ros2 topic echo /camera/camera_info --once ros2 topic echo /camera/detections --once

6. (Optional) Visualize images

rqt_image_view

7. Run nodes individually (alternative to launch)

Terminal A (video publisher):

source /opt/ros/humble/setup.bash source install/setup.bash ros2 run pennair_detect_ros video_node --ros-args -p video_path:=/ws/data/PennAirDynamicHard.mp4 -p scale:=0.5 -p frame_rate:=30.0 -p fx:=2564.3186869 -p fy:=2569.70273111 -p cx:=0.0 -p cy:=0.0

Terminal B (detector):

source /opt/ros/humble/setup.bash source install/setup.bash ros2 run pennair_detect_ros detect_node --ros-args
-p image_topic:=/camera/image_raw
-p out_image_topic:=/camera/detections_image
-p centers_topic:=/camera/detections
-p min_area:=600