More Vista and Nikon Camera RAW

Since the hubbub about incompatibilities between Nikons NEF codec in Vista and Photoshop, Microsoft has taken the automatic link to the Nikon codec off line.

In the mean time Nikon has issued new version in an attempt to address the compatibility problems with NEFs whose tags or metadata gets modified in Vista's Photo Gallery or Windows Photo Info.

http://nikonimglib.com/nefcodec/

But files modified by Nikon's new 1.01 NEF codec are still broken for me in Photoshop Bridge CS2.

Opening the modified NEF files works fine with PS CS2 Adobe Camera RAW, Lightroom or Photoshop CS3.

It seems that Bridge (CS2 at least) doesn't actually use ACR for thumbnails and previews. Nikon's tweak didn't fix the problem for Bridge so it may still be broken in other RAW handling software too.

I don't like or want Nikon writing to my NEF files. But it's nice that I can see NEF thumbnails in the Explorer and Photo Gallery.

If you mark NEF files as read-only then the Nikon codec doesn't modify them.

The other main problem for me is that Vista doesn't see the tags in XMP sidecar files. Also it may be possible for Vista's search to find tags in XMP sidecar files since it's just XML. I've found that Lightroom can associate sidecar files with the correct NEF files (although it gives you an annoying dialog each time you try to open a file from the Explorer that is already in your library). Bridge CS3 Beta 2 does not seem to be able to associate XMP sidecar files with their NEF files.

I'm seriously considering integrating DNGs into my workflow as this would eventually allow me to see and search for tags in Vista or WDS under XP. But Adobe's DNG codec for Vista and Photo Info is not ready yet.

Vista and Nikon Camera RAW Files

It all started with…

Metadata mangling in Windows Vista

It sounded like this was actually a problem with Nikon's RAW codec for Vista which is an optional download and install from Nikon. (Vista's Photo Gallery will automatically take you to the Nikon website to find it if it finds .NEF files).

From the MS Photo Info tool FAQ:

"Important note to users of the Nikon RAW Codec for Windows: Microsoft has received reports of compatibility issues with Nikon NEF files after installing version 1.0 of Nikon's RAW codec posted in January 2007. Tagging the RAW files through Windows Vista or the Microsoft Photo Info tool after the codec is installed appears to cause these files to become unreadable in other applications, such as Adobe Photoshop. We have confirmed that these files can still be opened with Nikon Capture.

"Nikon and Microsoft are investigating the issue, and we will post an update when we have more information. In the meantime, Microsoft suggests that you exercise caution with your Nikon RAW files. If you plan on tagging them using Nikon's codec, make a backup of the file first, and verify that the tagged file continues to work with your other applications before proceeding.

"Tagging the file using Photo Info without the Nikon NEF codec installed appears to be safe."

If you don't install the Nikon RAW codec Vista's photo gallery and explorer does not recognize the NEF files as photo files it can view or edit. (Of course, your image processing software like Photoshop and Bridge etc. will still recognize .NEF files.) You can tell Vista not to remind you about the Nikon codec.

So I actually tried Nikon's RAW codec 1.0 for Vista.

Indeed if you use Windows Photo Gallery to modify the image metadata or tags those NEF files become damaged and unreadable in obscure programs like Photoshop.

When you first access a folder with NEF files or other RAW formats in Windows Photo Gallery a dialog informs you that you can download software to access these files. If you click ok it takes you either directly to the vendor's download site for the codec or to a proxy. (For example, the DNG codec is not ready so eventually you get to a dead end that says "not ready".)

I downloaded and installed the codec from Nikon. (Actually due to a flaw in Nikon's download unpackager I was almost unable to install the codec because it unpackaged the files to some apparently random place on your drive and then doesn't start the installer nor give you the location of the files. So the actual installer gets lost on your drive.)

The install also requires you to reboot your computer which is very unusual under Vista.

After backing up a folder full of NEF files from my D70 I tagged a dozen or so files in Windows Photo Gallery (which is actually a nice simple photo browser).

The NEF files I tagged where then garbage in Photoshop CS3 (actually Adobe Camera RAW) and in Bridge CS2. Interestingly, Bridge CS3 was able to view the tagged NEF files. But Adobe Camera RAW invoked from Bridge CS3 could not. The files that I did not modify with tags were undamaged and could be read by Adobe Camera RAW.

Fortunately you can easily uninstall the Nikon Camera RAW 1.0 codec and no more damage is done. (Of course the tagged files are still mangled.)

I don't know if this is a continuation of the fight between Adobe and Nikon with respect to proprietary nature of Camera RAW data. It is clearly a dumb thing to allow any software to modify RAW files. Nikon should remove this codec from distribution until Nikon and Microsoft can fix the problems.

I don't see any way it can be considered a Photoshop bug even if Photoshop is doing something incorrectly in their parsing of NEF data. Photoshop is an existing compatibility constraint in this situation. A new program, especially one that places itself in a central roll in your image processing workflow, may not come along and break compatibility with Photoshop (by modifying and damaging your data) unless they have one hell of a good excuse. I have not read of any such excuse.

And apparently it's not just a problem in Photoshop. I've read that other applications (Bibble Pro 4.9.5, Photoshop CS2 Camera RAW 3.6, RawShooter and DXO) have the same results with the Nikon codec.

It's a dumb idea because if you allow just anyone to come along and modify your RAW files you get situations exactly like this. You would think you could trust Nikon, the maker of the camera, not to mangle your data. But that is clearly not the case. So trust no one!

Many photographers wisely consider Camera RAW files to be somewhat analogous to negatives. Likewise Adobe considers Camera RAW files to be read only. When you modify tags, copyrights and other metadata in Adobe's Camera RAW filter, in Adobe Bridge or in Lightroom, Adobe stores the modified and new metadata in a XML file (called an XMP sidecar file) or in a central database. The XMP sidecar file is an inelegant solution because it means you have two files that have to be moved around together. But it allows for interoperability between different applications. And it is far less inelegant than having your image files inadvertently munged and your workflow rendered inoperable.

The really pernicious thing about this bungled piece of software is that it could destroy your image catalog if you were doing something seemingly innocuous like importing RAW files from a memory card using the OS's batch import wizard or using the Windows Explorer or Windows Photo Info Tool to update your copyright notice in a folder full of files.

The only other application I know of that modifies Camera RAW files directly is Photo Mechanic. But photographers who use it that way tend to know exactly what it is doing and why. As I recall it warns you, and you have to explicitly tell it to modify the RAW file through a preference.

Apparently the data is not damaged per se. It just is no longer readable by Photoshop et al until they update their Camera RAW filters. (That might explain why Bridge CS3 can read the files but PS CS3 and PS CS2 cannot.)

I'm thinking it would be a good idea to add a step to my workflow to set the read-only bit for all RAW files and other original files.

Using Adobe's DNG seems like a good idea. But I'm not too keen to add another step to my workflow.

More on Lightroom 1.0

I haven't yet figured out the best way to share an image library between different computers.

According to Adobe's Lightroom FAQ:
"Will my Lightroom Library synchronize with different computers and networks, or am I limited to one only?
"Lightroom shares a limited amount of information using metadata stored in XMP format. Lightroom plans to add more robust support for multiple computer environments in future versions."

I am pretty impressed how well Lightroom accommodates my existingworkflow and organization including storage locations and file naming schemes. My image file organization system divides original and derivative files into two major storage branches. It's clear that under the Lightroom model (and the Bridge/ACR system for that matter) that organization may need some adjustment. Or, at least, I will need to think about the term "original" differently since changes are constantly added to the XMP data accompanying "original" files.

As I noted, Lightroom is quite adept at dealing with files that are located on a server or other computer, and it can deal with files coming and going gracefully. But it's not really designed to support multiple computers or users working on the same file (never mind the obviously complex problems of simultaneous access). It stores your net Lightroom adjustments in the XMP data for the file (either in a sidecar file for RAW or in the metadata of the file itself for JPEG and DNG). There is a preference to always write XMP data to the image files or it can be done manually through a menu. But that data seems to be a single snapshot. Your adjustment history, for example, seems stored in the Lightroom library on your computer.

There may be a roll for Windows' offline files/sync center feature. But I tread carefully.

Adobe Photoshop Lightroom 1.0

I'm going to gush. Lightroom 1.0 is brilliant.

When Adobe added Bridge to Photoshop CS2 I found I could process a batch of RAW files in an hour or two where previously it had taken me all day. It was a dramatic productivity improvement that made shooting Camera RAW viable and allowed me the benefit of the format's inherent advantages.

What I've seen in the few days I've been using final release of Lightroom 1.0 promises similar revolutionary improvements in the photo productivity. It's that good.

There are three keys to this -- the caching system, non-destructive editing and the complete workflow from camera import to final destination output.

I found the Beta versions of Lightroom frustrating because performance was not well tuned, the caching system was incomplete (on Windows at least), and I kept running into incomplete and unfinished features. The final version has progressed a lot since last fall's final beta. Performance is outstanding and all the features seem well done. I think the open beta process is to thank for the latter. I've been using Lightroom on a two-year-old 3.6GHz P4 3GB XP system and a six-month-old 2GHz Core Duo laptop running Vista with 2GB.

A good example of Lightroom's caching efficiency is working with files on a server. Using images stored on a RAID 5 NAS is surprisingly usable over both Ethernet (Gb) and 802.11g. The cache system works amazingly well for both 5MP JPEGs and 6MP D70 RAW files. For example, Lightroom allows you to start working on an image in the develop module while the actual file is loading in the background. And you can even manage, but not develop files that are not even physically present such as when you are not connected to a LAN or images that are stored on a removable drive. This makes work on a laptop quite viable.

Virtual Copy is an amazingly simple mechanism that allows you to work on different versions of the same image. Just pick an image in your library and choose Create Virtual Copy. Now you have two images based on the same file. This works because Lightroom doesn't actually change the pixels in the source image.

Lightroom's keyword and metadata tagging is what I've been looking for. Fast caching (it works in the background while you work at your own speed) combined with a range of options: drag-and-drop, stamping, typing make the usually painful task easy and maybe even fun.

Lightroom's filtering is a reasonable compromise that's fast and accessible. It's far better than just a single term search since you can do some Boolean operations (disguised with terms like find "Containing All" or "Not Containing"), and you can combine those finds with date ranges and other filter options like folders, collections, picks, ratings. But, the find and filter mechanism is not infinitely flexible. For example, I've not figured out how to search for images that have no keywords.

Color management in Lightroom is transparent (finally, transparent color management in a Photoshop product!). You still are responsible to calibrate your monitor and to understand how to either turn off color management in your printer driver and have Lightroom handle color or turn off color management for printing in Lightroom and have your printer manage it.

Lightroom includes both Mac and Windows versions -- a nice touch in the age of Boot Camp. Adobe's license allows you to use your software on a second portable or home computer but not at the same time.

I'm glad I don't have to decide between Aperture and Lightroom. The programs seem very competitive, but I don't see anything in Aperture I'm pining for. The competition is a good thing for the customers since it will drive the product category even farther.

After using the release version of Lightroom I finally understand why Adobe is producing both Lightroom and Photoshop CS3 with many of the same abilities. It's because they are quite different programs that share a common Adobe Camera RAW technology. With Lightroom, Adobe was able to look at the problems of processing photographic images and create a product that works without all the constraints of Photoshop's 15 years of baggage. To some extent Bridge 1.0 did that with its Camera RAW processing abilities. But Bridge really only allows you to batch process your RAW settings. You still have to generate JPEG files or print through Photoshop (or Photoshop's image processing mode). Lightroom brings that rethink to a different level allowing you keep your images in their original format and apply your processing non-destructively all the way up to your final destination most of the time.

If you need to leave the world of Lightroom for a broader range of processing options, Photoshop (or the editor of your choice) is right there and can work with your images at the click of menu. But taking a photo into Photoshop means leaving the Lightroom workflow and in some cases making a branch off the Lightroom edits. If you are working with JPEG, TIFF or PSD files in Lightroom you have the option of editing the original image or a copy of the original pre-Lightroom adjustments or a copy with Lightroom adjustments. For RAW and DNG files you must edit a branch version of the image with all your previous Lightroom non-destructive adjustments committed. In essence we are back to something like Bridge and Photoshop with Lightroom filling in for Bridge.

I'm sure there are flaws. But I'm still on my honeymoon with this program, but I'm very happy so far.