Month: September 2020

I’ve been a loyal TiVo customer for more than 16 years which feels awkward to type. TiVo never really cracked into the mainstream, it has always been a premium product and it has since gone thru many acquisitions and CEOs in recent years. I currently own a TiVo Bolt and several TivoHDs. The TiVo is a DVR (Digital Video Recorder) which is a market they practically invented along with a competitor named ReplayTV. I’m not a heavy television watcher but I do watch sports and particularly NHL Hockey. The TiVo can setup a “season pass” on a particular sports team and record all games (i.e. Dallas Stars). My TiVo Bolt has a 500 GB internal HD which is capable of holding many dozens of shows; the Dallas Stars are on a historic Stanley Cup run and I wanted to archive some games for watching later / posterity. The TiVo has native Plex App support so I wondered if I could pull the recorded MPEG shows off the TiVo and archive them in a Sports Plex Library on my FreeNAS server (which has many TBs of storage and ZFS!).

Update 25-Oct-20: The Dallas Stars lost the cup to the Tampa Bay Lightning in 6 games. It was exciting hockey and I’m sure glad I have the series on my Plex. I’m excited for next season, looks like we’ve signed most the team back with the exception of Corey Perry and Matthias Janmark.

Step 1: The Fair Use Doctrine

Am I legally allowed to record and save a show from my TiVo to the Plex? Accord to the Fair Use Doctrine the answer is yes.

I can legally record a broadcast TV program (from a Cable TV feed I pay a monthly fee for) on a recording device such as a Video Cassette Recorder, Digital Video Recorder or computer for my personal use. Recording programming for later viewing is called “Time Shifting” and is a legally supported act. I can view the recorded content as many times as I want, I can make additional copies of the recorded content and I can distribute copies of the content to other people in person. Educational uses of recorded TV content receive additional protection. It is legal to record a TV program and play it for a class. I can also edit and parody the recorded content as long as the new version adds new value (insight, understanding, aesthetics) to the original. 

I can’t hold public exhibitions, rebroadcast, distribute on peer-to-peer networks or sell a TV broadcast recording. Fair use does not protect mass distribution of recorded content or the ability for a person to make a financial profit from a copyrighted recording.

The TV recording copyright laws for the U.S. are rooted in the Sony Corporation V. Universal City Studios court case–commonly known as the beta max case. The Supreme Court determined that “video tape recorders” did not infringe on the TV programming copyrights and the potential financial loss to copyright holders from TV recording would be negligible. The case made it legal to produce and sell TV recording devices and for people in the U.S. to record TV programming.

Step 2: How Do I get my Shows off the TiVo?

Let me introduce you to a wonderful little piece of software called pyTivo. This open source software which is not affiliated in any way with TiVo, Inc provides similar functionality to the TiVo Desktop software for Micro$oft Windows and Apple Macintosh. Namely this software allowed you to push and pull MPEG video files from my TiVo and I have been a happy user for over 10 years. The software runs on a host (or a NAS or wifi-router) on your local area network and listens via multi-cast IP for connected TiVo devices. The software is very simple to install and very light requiring only Python, ffmpeg (to transcode) and the tivodecode utility to decrypt TiVo video files. I setup a Jail on my FreeNAS 11.3 server where I wanted to install pyTivo. I had previously setup a share called sports on my medialibrary directory on my NAS pool. I kept the sports folder seperate from Movies or Television on the Plex and in the medialibrary since Plex expects a certain naming structure for Television shows or Movies. I installed the required software:

rich@kidney$ pkg add ffmpeg git tivodecode
rich@kidney$ cd ~/
rich@kidney$ git clone git://repo.or.cz/pyTivo/wmcbrine.git
rich@kidney$ mv wmcbrine pyTivo
rich@kidney$ cd pyTivo
rich@kidney$ cp pyTivo.conf.dist pyTivo.conf
rich@kidney$ vi pyTivo.conf
rich@kidney$ ./pyTivo.py

Step 3: Configuration of pyTivo and First Download

Jot down the IP address of the Jail you configured in Step 2. Fire up your favorite web browser and head over to that address and port 9032 (which is the default, you can change this). The pyTivo user interface is very simplistic. It will initially only show you a Settings link which you must configure for first use.

I’ve circled the most important settings and how I’ve configured them. The paths to ffmpeg and tivodecode and tdcat binaries must resolve. You can test the fully qualified path names to those binaries on the FreeNAS ssh command line. Please note that I compiled a fresh version of tdcat and tivodecode from wcmcbrines git repo as the stock FreeBSD tivodecode binary seg faulted for me. I set my beacon address to the multi-cast for my local subnet and I ensured that my FreeNAS box was on the same local subnet as my TiVos. The togo_path is important. This is where all shows downloaded from the TiVo will be saved local to the pyTivo installation. As previously mentioned I created a sports mount point to my medialibrary share on my ZFS pool. Lastly, I filled in the TiVo MAK (Media Access Key) for my particular account. This is a secret key used to decrypt shows recorded on my personal TiVo — Do not share this key! You can find your MAK on tivo.com when you login to your account. Save changes and restart your pyTivo to ensure changes take affect. Make sure on the command line that pyTivo.conf has been updated with the information you input in the Settings menu. When you restart pyTivo you should see a listing of your TiVo’s and if you input a correct MAK key then you should be able to connect and browse a listing of shows on the Tivo.

If you click on a folder you should see a listing of shows with a description of each one along with the runtime, the file size and the date it was recorded. The exclamation point next to each show means these shows will be deleted shortly as needed for space. Remember: I only have a 500 GB model…

So, each Dallas Stars hockey game is 3.5 to 4 hours and it’s about 7 GB of 720p video recorded at 4 Mbs. I can of course play around with how this is recorded on the TiVo Bolt itself but this seems to be a good compromise of space/size vs. quality. Honestly I want to avoid any transcoding and just focus on decrypting and downloading to the NAS.

Click the check next to each show you want to download and check the bottom boxes to decrypt, save meta data and transfer as mpeg-ts. As far as I understand the mpeg-ts doesn’t transcode and just copies the raw file from the TiVo.

Rename each file and add .mpg extension; the text file will contain meta data for the video.

Quick Update 23-Sep-20: You don’t need to rename the file extension to .mpg for the Plex file scanner to “see” the video files. Just make sure the file permissions allow Plex at minimum to read the file. A simple check of chmod 777 could help spot if that is the issue. The file will come into Plex with the default filename as the Title. You can “enrich” this information manually in the Plex UI if you wish. Plex seem to take 3 or so stills from the video file to use as the poster in Plex. These seem to be fine for me in most cases.

Step 4: Add Tivo function user to run PyTivo Jail

In the FreeNAS web console, select Accounts and Users and click on the Add button. Fill in the new user details, in my case I created the user tivo and most important set the UID (user id) to a unique number higher than 1000. In my case I chose uid 1010. Next set the group for the tivo function user to something that you use with your media, in my case the media group. You can setup a home directory if you wish (it’s not necessary). Click save to create that user.

Ssh into your tivo FreeNAS Jail using the following command

root@kidney$ jls tivo
root@kidney$ sudo jexec 8 /bin/tcsh  <<whereas 8 is your tivo jail id>>

root@tivo$ whoami

Now that you are in your Tivo jail as a the root user which you can see from the command line. Users and groups created within a FreeNAS jail are only visible in that jail. Meaning you can create users here but they are unique to that jail and are not in your overall FreeNAS system. But, you can create a new user in the jail and use the same UID and GID as the users and groups you setup in the overall FreeNAS system. That way any file saved will show tivo and group media in both the Jail and in FreeNAS and via shares. To do this run the following commands within the Tivo jail:

root@tivo:/mnt/sports # pw groupadd media -g 8675309
root@tivo:/mnt/sports # pw groupshow media
media:*:8675309:


root@tivo:/mnt/sports # adduser
Username: tivo
Full name: TiVo Functional User
Uid (Leave empty for default): 1010
Login group [tivo]: media
Login group is media. Invite tivo into other groups? []: 
Login class [default]: 
Shell (sh csh tcsh git-shell nologin) [sh]: 
Home directory [/home/tivo]: 
Home directory permissions (Leave empty for default): 
Use password-based authentication? [yes]: 
Use an empty password? (yes/no) [no]: 
Use a random password? (yes/no) [no]: 
Enter password: 
Enter password again: 
Lock out the account after creation? [no]: 
Username   : tivo
Password   : *****
Full Name  : TiVo Functional User
Uid        : 1010
Class      : 
Groups     : media 
Home       : /home/tivo
Home Mode  : 
Shell      : /bin/sh
Locked     : no
OK? (yes/no): yes
adduser: INFO: Successfully added (tivo) to the user database.
Add another user? (yes/no): no
Goodbye!

In the above instance I had already created a media group in FreeNAS with 8675309 as the group id (I guess I was in a comical mood that day! Jenny Jenny … lol ).

Step 5: Setting pyTivo process to Autostart as tivo user

Update: 25-Oct-20: This one took me a bit of time to figure out. I followed this guide which was written for another python program called octoprint. In the guide they talk about writing an rc script to start the program and in particular to run it as a daemon. I did a bit of tinkering and I got it to work with pyTivo. Please note that pyTivo currently still needs python 2.7.

!/bin/sh
 PROVIDE: pytivod
 REQUIRE: DAEMON LOGIN NETWORKING
 KEYWORD: shutdown
 . "/etc/rc.subr"
 base_path="/usr/local/bin"
 name="pyTivo"
 desc="pytivo daemon"
 rcvar="pytivod_enable"
 pidfile="/var/run/${name}.pid"
 command="$location/$name/$name.py"
 command="/usr/sbin/daemon"
 command_args="-p ${pidfile} -u tivo -f -c ${base_path}/${name}/${name}.py"
 command_interpreter="/usr/local/bin/python2.7"
 load_rc_config $name
 run_rc_command "$1"

The rc script with the above content was saved in /usr/local/etc/rc.d as pytivod and I made sure to set the script as executable. I also made sure that pytivod_enable=”YES” is in the /etc/rc.conf file within the Jail. Restart the Jail and check that the process has auto-started and that the process is running as the tivo user. This is really, really important for me as I want all files to be owned by tivo:media on the TrueNAS and I can set the plex user and the tivo user to be in the media group so they have read/write access to those files.

Step 6: Add Plex Library for Other Videos

In your Plex web user interface add a new Video Library, make sure you choose other videos. I named the library sports for NHL and MLB games. In the Plex user interface you can scan for new media files and update the title and other details from the meta data text file. In the Other Videos category it’s really very simple and it doesn’t scan and enrich the filenames. I will need to add a renamer function to add series name and episode number to the filename of the files if I want a more seamless “Television Shows” experience on Plex. I’m still working on that part and I’ll update this posting when I get somewhere with that.