Script: Rename images by EXIF date
My earlier post on photo organizing in linux contained a script I wrote to rename pictures based on the EXIF date. However, since I’ve been using F-spot and upgrading the OS and copying files here and there, I still, despite my best efforts, managed to accumulate duplicate images. Unfortunately gqview was not able to detect these with my trusted checksum duplicate detection scheme, and it was too much a hassle to try looking at dupes based on similarity. I think part of the problem was that I imported photos into F-spot more than once. Some images, then, had tags, while others did not. I assume that’s why the checksums for duplicated photos were different.
In any event, I decided to rename all of my pictures based on the EXIF date, right down to the second. This produces filenames like YYYY-MM-DD-HH-mm-ss.jpg, e.g. 2007-05-15-21-30-17.jpg. I don’t take a lot of pictures in rapid-fire mode, so most of mine can be distinguished using the second field.
In case you do use rapid-fire mode on your camera, or you have multiple versions of a picture containing the same EXIF date, the script won’t overwrite existing filenames; instead, it appends “_copy” to subsequent filenames. Yes, you could end up with files like 2007-05-15-21-30-17.jpg, 2007-05-15-21-30-17_copy.jpg, 2007-05-15-21-30-17_copy_copy.jpg, etc. Better to be safe than sorry, I say. Just to be a little safer, I copy the pictures to a separate directory, ./exif/YYYY.
All in all, this works pretty well. I got all of my pictures sorted by year and reimported them to F-spot. Now I can hunt for file names containing “_copy” and better determine which version I want to keep.
I call the script WMDD (Weapon of Mass Duplicate Detection), which might be a little misleading because it doesn’t actually detect duplicates. I had just watched a Frontline on the domestic surveillance program, so I had dubya on my mind.
BTW, there’s a patch for duplicate detection in F-spot floating around somewhere, but I don’t think it’s made it in the main release yet.
June 27th, 2007 - 3:09 pm
Hi Todd
Could you specify which “exif package” is required by WMDD. I’d like to try the script.
Thanks. Chris
July 18th, 2007 - 6:14 am
dude, wouldn’t this be easier?
jhead -autorot -nf%y%m%d-%H%M%S *
July 18th, 2007 - 7:51 am
dude, you’re right! Been awhile since I used jhead and I forgot it could easily rename images. I was extra paranoid about overwriting files during the rename process, and after investigation, I see that jhead appends letters [a-z] to the end of filenames that would otherwise be duplicates.
September 10th, 2007 - 2:32 pm
dude, how does jhead handle files without metadata? and will jhead add the “_copy” to the file name if the metadata is duplicate? I have files from 4 cameras in a PHOTOs folder…
Thanks,
Paul
September 10th, 2007 - 3:03 pm
@ Paul dude, if there’s no valid DateTimeOriginal in the exif header, jhead uses the file date. If DateTimeOriginal is a duplicate (file exists), jhead appends a letter [a-z], e.g. 2007-09-10.jpg, 2007-09-10a.jpg, etc. Of course you specify how you want jhead to rename the files as Michael pointed out.
December 19th, 2007 - 2:34 pm
I am a bit new to this, so please excuse me if my question is “dumb”. Is the suggestion “jhead -autorot -nf%y%m%d-%H%M%S *” meant to replace all or just some part of your WMDD script, and if so, what? I did try it, and it renamed the photos OK.
I tried WMDD, but obviously either I or it did something wrong as it put my photos into the noexif file and gave the message: “IFD ‘(null)’ does not contain tag ‘DateTimeOriginal’”. However, the photos DO have a creation date in the EXIF metadata. If I run exif on one of the images, it shows the exif data including “0×9003 Date and Time (original)”.
Can you offer any suggestions?
Next, what I would REALLY like the new names to be something like:
YYYY-MM-DD-HH-mm-ss_.jpg but I would like to get your script working before I try to modify it to that. I really like the idea of putting the output photos into two new folders exif and noexif.
Thanks for any ideas.
December 19th, 2007 - 3:56 pm
@alpha1 the jhead command will replace the whole script; it renames images based on the values for year, month, day, etc. You can specify the format by the way you order it. “man jhead” or “jhead –help” should give you some ideas. So in the example, you can do something like %y-%m-%d etc.
December 19th, 2007 - 5:03 pm
todd: I can see that jhead replaces the names. However, what I like about the description of the WMDD script is that it leaves the original photos alone and then sorts the rest into two new folders (exif and noexif) with the new names. The next thing would be to append the original image label to the resulting image. E.g. John_Christmas.jpg would then become something like:
071214-172238-John_Christmas.jpg — I just don’t know how to do that but given time might be able to figure it out. The only problem is that I have not been able to get WMDD to work. This would result in photos being arranged by date, but all those that I have labelled would retain their labels so that I would have an easier time recognizing what they are.
December 19th, 2007 - 5:29 pm
alpha1: can you post the results of the command ‘exif -t DateTimeOriginal <somepicture.jpg>’ where somepicture.jpg is one of your images?
December 19th, 2007 - 7:26 pm
exif -t DateTimeOriginal
yields:
IFD ‘(null)’ does not contain tag ‘DateTimeOriginal’.
for a photo for which jhead -autorot -nf%y%m%d-%H%M%S changes the name of which was taken on Dec. 10 2007 at 17:22:38 to:
071210-172238.jpg
December 19th, 2007 - 7:29 pm
OOPS! That should have been:
for a photo for which
jhead -autorot -nf%y%m%d-%H%M%S .jpg changes the name of
.jpg which was taken on Dec. 10 2007 at 17:22:38 to:
071210-172238.jpg
December 19th, 2007 - 7:31 pm
still got it wrong!
That should have been:
for a photo for which
jhead -autorot -nf%y%m%d-%H%M%S .jpg
changes the name of
.jpg which was taken on Dec. 10 2007 at 17:22:38 to:
071210-172238.jpg
December 19th, 2007 - 7:33 pm
For some reason, this deletes the less-than-sign some picture greater-than-sign everywhere that I typed it before .jpg
December 19th, 2007 - 8:55 pm
hmm, I’m getting that problem too. I think it’s just a matter of changing it from DateTimeOriginal to DateTime. If you run just “exif picture.jpg” it should print out all the fields; check for the one that gives the date and time. It looks like my photo manager inserted exif fields for when an image was imported–weird.
December 19th, 2007 - 9:00 pm
Let’s see if this works. I think I need the quotes around stuff which contains things that might be interpreted as HTML tags
(such as ”).
‘exif -t DateTimeOriginal .jpg’
yields:
IFD ‘(null)’ does not contain tag ‘DateTimeOriginal’.
for a photo for which
‘jhead -autorot -nf%y%m%d-%H%M%S .jpg’
changes the name of ‘.jpg’ which was taken on Dec. 10 2007 at 17:22:38 to:
071210-172238.jpg
December 20th, 2007 - 9:32 am
Todd, if I just run exif on the image, the appropriate line shows up as:
Date and Time (origi|2007:12:10 17:22:38
I think I see the cause of the problem!
If I do exif -i the appropriate line has:
0×9003 Date and Time (original) gives:
EXIF tags in ‘071210-172238.jpg’: 0 1 EXIF GPS Interop
.
.
.
0×9003 Date and Time (original) - - * - -
I assume the * indicates where the information is contained. In this case, the Date and Time (original) information is contained in EXIF. The WMDD script must be looking in field 0, which would probably be IFD ‘(null)’ instead of the EXIF field. I guess jhead must look at the EXIF entries rather than or as well as IFD0.
I also think I seed why the ” was not showing up my earlier posts; When I type a single quote, it inserted ‘ not ‘ .
There is a DateTime entry in IFD0 (0×0132) and if I change DateTimeOriginal to DateTime in the script, it does sort of work — except that DateTime gives the date and time the image was last changed, not created, i.e. 2007-12-18-12-23-52.jpg, not 2007-10-12-17-22-38.
I also figured out how to append the original name so that now a photo originally named orig_photo.jpg gets the name:
2007-12-18–12:23:52-orig_photo.jpg
Now if I can only figure out how to get the date taken/created information from the EXIF field instead of the DateTime from the IFD0 field, I will have what I wanted.
Here is the script, as I have modified it to this point:
#!/bin/bash
#
# wmdd - Weapon of Mass Duplicate Detection
#
# I wrote this because I was having a hard time identifying duplicate
# images in my f-spot Photos folder. I decided I should just rename all
# of the pictures based on the EXIF data to YYYY-MM-DD-HH-mm-SS.jpg format.
# If pictures have identical EXIF data, then ‘_copy’ is added to the file
# name. Operation is recursive.
#
# To make it a little easier, I have the script copy pictures to a folder,
# ‘exif/YYYY’ or ‘noexif’ depending on whether there is EXIF info or not.
# This also make reimporting to f-spot a little easier.
#
# Requires exif package.
#
# To run, cd to the top-level directory of the images you want to rename.
#
# Todd Slater
# 14 May 2007
# modified by alpha1
# 20 December 2007
original_pic_dir=`pwd` # maybe at some point make this an optional argument
cp_dir=”$original_pic_dir/exif”
noexif_dir=”$original_pic_dir/noexif”
# find images
find $original_pic_dir -type f -iname ‘*.jpg’ >> piclist
while read line
do
# test for exif
if [ -z `exif -t DateTime “$line”` >/dev/null 2>&1 ] ; then
# no exif, let’s copy it to noexif_dir, but not overwrite existing files
pic_name=`echo “$line” |sed s/” “/_/g | xargs basename`
while [ -f “$noexif_dir/$pic_name” ] ;
do
pic_name=`echo “$pic_name” | sed s/\.jpg/_copy\.jpg/`
done
if [ ! -d $noexif_dir ] ; then
mkdir -p $noexif_dir
fi
cp “$line” “$noexif_dir/$pic_name”
else
# has exif, let’s rename and copy to exif/YYYY folder
pic_name=`echo “$line” |sed s/” “/_/g | xargs basename`
exif -t DateTime “$line” |grep Value > exif.$$
year=`cut -b 10-13 “exif.$$”`
month=`cut -b 15-16 “exif.$$”`
day=`cut -b 18-19 “exif.$$”`
hour=`cut -b 21-22 “exif.$$”`
minute=`cut -b 24-25 “exif.$$”`
second=`cut -b 27-28 “exif.$$”`
rm exif.$$
exif_name=”$year-$month-$day–$hour:$minute:$second-$pic_name”
while [ -f “$cp_dir/$year/$exif_name” ] ;
do
exif_name=`echo $exif_name | sed s/\.jpg/_copy\.jpg/`
done
if [ ! -d “$cp_dir/$year” ] ; then
mkdir -p “$cp_dir/$year”
fi
cp “$line” “$cp_dir/$year/$exif_name”
fi
done
December 21st, 2007 - 9:52 am
On my images, the DateTime tag is the date/time taken while DateTimeOriginal is the last modified. Hmm, gonna have to do some investigatin’.
January 9th, 2008 - 2:44 pm
> Now if I can only figure out how to get the date taken/created information
> from the EXIF field instead of the DateTime from the IFD0 field, I will
> have what I wanted.
try this:
exif -t DateTimeOriginal –ifd=EXIF
January 25th, 2008 - 3:53 pm
After running the command “sh ./wmdd.sh” in the folder with images I get this error: :
“not found21: ”
“./wmdd.sh: 61: Syntax error: “done” unexpected (expecting “then”)”.
What is wrong?
January 25th, 2008 - 4:19 pm
Off the top of my head I’d guess it has something to do with spaces in the file name. I’ll have to investigate, though.
January 25th, 2008 - 5:09 pm
todd, in my ‘piclist
‘ are listed all images with spaces in the file name.
P.S. I run Ubuntu, maybe this is the problem?
January 26th, 2008 - 8:24 am
todd, I finded this links in Spanish, maybe it will be useful: _http://www.gilug.org/?q=node/223
_ftp://copernic.udg.es/pub/festuc/orgafotos
January 28th, 2008 - 3:10 pm
orgafotos works for me. I only replaced code ‘0×9003′ with ‘DateTime’. The only incomodity are some messages in Spanish, but this is not a big problem.
January 28th, 2008 - 3:28 pm
P.S. I forget to mention: I used the adapted script from here
May 21st, 2008 - 2:57 am
I’m very happy if anyone more use my orgafotos too.
If need some trasnlation i can make with a lot of ortografic faults, if you can correct me, we can make a translation
May 21st, 2008 - 2:59 am
My script don’t rename files, just put files into a structured directory list ordered for year, and day
May 21st, 2008 - 3:44 am
Original code of orgafotos is in catalan …
May 21st, 2008 - 4:07 am
My bad english version of my orgafotos:
http://festuc.info/orgafotos
September 14th, 2008 - 4:36 pm
Great minds think alike - many people seem to create scripts for organizing their pictures by date and time. So did I. I wrote a script to replace the F-Spot script that is automatically called (by default) when a camera is connected in Ubuntu. My script retrieves the pictures from a camera that is mounted as USB Mass Storage, renames them to, for instance, P20080914-2333.jpg and puts them in a year-month directory like ~/photo/2008-09/, which is created if it doesn’t exist. If a file would be overwritten, no action is taken, but it is reported so that you can consider what to do. Most error conditions are taken care of; Xdialog is used to communicate with the user, including a progress bar showing the picture transfer process. That means, that it can also be used by non-CLI users, via an icon on the Desktop. Actually, that was my primary goal - initially to free ordinary users from mounting camera storage, etc., later to prevent the pictures of such users to fall in the hands of F-Spot, which works in such a way that the creation of duplicates is very likely, which puts pictures in a directory structure that hinders clear overview and uses a database(!) to keep an overview itself.
(I mentioned Ubuntu, but the script can, of course be used in any Unix/Linux environment - initially, I created it on a FreeBSD system.)
If there is any interest in my script, I’m happy to share it, either by mailing it or by making it available via some website.
September 17th, 2008 - 12:17 pm
Thanks for the note, @Menno, I’m discovering more issues with the way F-spot handles date/time and it’s become fairly annoying. It adjusts the time by +4 hours on my images. The bad news is that I previously allowed it to write metadata to the image files, and so it changed it in the images; the good news is that there’s still an exif field that retains the correct date and time stamp. The bad news it means running yet another script to correct the problem, and it’s just getting very annoying. I’m sure others would appreciate a look at your script.
September 17th, 2008 - 7:00 pm
My script for organizing pictures as well as some related scripts are available via http://www.tjoelker.org/scripts.html
I hope that it is useful for somebody.
December 21st, 2008 - 8:19 am
Just looking back at my earlier posting with the modified
WMDD script, I had not posted the version that actually
works. (I cannot believe that a full year has passed in the
meantime!) In the script above, replace the following line:
exif -t DateTime “$line” |grep Value > exif.$$
with this one:
exif -i “$line” | grep 0×9003 > exif.$$
which finds the tag 0×9003, the correct one for DateTimeOriginal, or the creation date.