Was trying to locate the source of an issue that came in in the tickets and it took me a while, so putting the solution out there for the community in case anyone else runs into it.
The issue was that after upgrading to Jamrom 6.1 all the videos would not show up where they were expected to. The video player playlist was just blank where there should have been videos.
The solution is to use a new tool in the videos module:
ACP -> MODULES -> PROFILES -> VIDEO -> TOOLS -> VERIFY VIDEO FILES
What has happened is that going from Jamroom 6 to 6.1 the default video format has changed. Previously the default was .flv and the fallback was .mp4. From Jamroom 6.1 this has reversed. So to make sure there is a video file in the default format to be played this new tool was added.