HD News: Don’t Fear the File!, by Ed Casaccia

The winners in the toughest ratings fight have one thing in common: Thomson Grass Valley™ multiformat and HD digital news production systems.

They're installed at leading station groups in the top five U.S. markets -- and at leading news outlets worldwide. With a complete workflow that spans acquisition, production and playout, it's clear that when you're watching the news, you're watching Grass Valley technology at work.

For more information about Thomson Grass Valley products, please visit thomsongrassvalley.com.

Free White Papers

HD News: Don’t Fear the File!

by Ed Casaccia, November 15, 2006

Director of Product Management and Marketing, Digital News Production, Grass Valley

OVERVIEW

The era of HD news is upon us, later than the earliest visionaries may have forecast, but also more suddenly than many expected. Nine years and two months after the Balanced Budget Act of 1997 defined the digital television landscape for the U.S., and four years and a month after the originally mandated date for auction of analog channels, HD news has in the past few months become a key marketing differentiator for local news offerings in large U.S. television markets.

This trend, apparently driven by a “critical mass” being reached in HD home receiver penetration, is part of the feedback effect in HD adoption: more programming spurs more consumer purchases of HDTV which spurs the creation of more programming. This phenomenon is echoed in other areas of the world, particularly in the wake of this year’s widespread HDTV distribution of World Cup soccer coverage. In fact, high definition news production systems are now installed or being installed in North America, Europe, Asia, and Oceania.

There is one key characteristic that differentiates true HD news operations from “studio only” HD news with 16:9 standard definition field acquisition: while videotape HD acquisition formats such as HD-Cam and HDV certainly exist, all of the practical and professional HD news field acquisition systems involve writing files to non-linear removable media. Put another way, the HD news workflow is a file-based workflow. The implications of this fundamental fact are the basis for this paper, and raise the following list of issues that must be dealt with by the file-based production system.

Multiple aspect ratios in SD/HD transition

Multiple HD resolution formats

Multiple codecs

Multiple wrapper types

Multiple metadata standards, (including those specified by wrapper types)

Inter-system compatibility

These issues all point to one goal for any HD news production system. That goal, strangely enough, can best be phrased in terms of describing what used to be the expected norm. That HD news production systems must provide the same ease of integration and transparency of interoperability that was universal when the BNC was the only signal connector, analog composite was the signal type, and RS-422 provided the control layer. The plethora of different file types, picture formats, storage and infrastructure architectures, and control interfaces has introduced an unnecessary element of fear into the HD news transition.

Multiple Aspect Ratios

We can begin with the issue that most newsrooms will encounter first. That issue is mixing aspect ratios among a wide variety of source material and/or for two or more different outputs. Most newsrooms will begin dealing with this the moment they begin acquiring material in 16:9 aspect ratio, whether SD or HD. The sheer quantity of 4:3 archive material possessed by most newsrooms dictates that some form of aspect ratio conversion will remain part of the workflow for the foreseeable future, since only the largest and wealthiest operators can even consider converting their entire archive to 16:9 in situ. Note that for the vast majority of newsrooms, the aspect ratio conversion will be a one-way street, requiring some means to adapt 4:3 material into a 16:9 frame, but late adopters will have to do the opposite when they are among the few still shooting 4:3 in a world of pools and syndicated news feeds in 16:9.

Since virtually all camcorders used in news for the past 5 years have been capable of selecting aspect ratios on a story-by-story or shot-by-shot basis, aspect ratio conversion first becomes an issue in the editing process. This is the first point at which the file-based news production system must offer solutions.

Nonlinear editors are capable, and should offer all, of the following aspect ratio conversion techniques as standard capability without extra charge and transparently to the user. Further, they must offer the ability for newsroom management to select a standard technique that is automatically applied but which can be over-ridden on a shot-by-shot basis for esthetic reasons. All of the following techniques should be supported in the editor.

Anamorphic Stretch

(click thumbnail)

Technique one is the easiest to accomplish, but perhaps the most obtrusive from a viewer standpoint. In this, the 4:3 image is stretched horizontally to fill the 16:9 frame. This anamorphic stretch can take place unintentionally if a 4:3 file is not appropriately identified and reaches a server playout port set to deliver material in 16:9. This technique distorts the image, changing circles into ellipses with the major axis in the horizontal screen dimension, thus:

Pillar Box

Pillar Boxing, maintaining the 4:3 image as is and filling out the width of the 16:9 frame with some other video material has become the de facto standard in the U.S.

(click thumbnail)

Its great advantage is that it is distortion free other than any up conversion artifacts when going from 4:3 SD to 16:9 HD. It has the significant disadvantage that consumer HDTV receiver user manuals generally caution against keeping such a converted source on screen for a long period of time due to burn-in concerns, particularly on plasma screens. Due to that concern, some home viewers engage their set’s aspect ratio conversion capabilities to zoom in to the image, losing many lines of vertical information but filling the horizontal dimension.

The non-linear editor must be able to overcome that objection by:

a) Filling the pillars with superblack allowing the production switcher to fill with an animation or other moving image.

b) Allowing a standard graphic or animation to be automatically added during a transparent conversion process within the NLE.

There is also a compromised method referred to as “Half Bar” in which the 4:3 image is distorted slightly to 14:9 then inserted into the 16:9 frame, leaving narrower bars.

Interestingly, the end-user practice of zooming past the pillars is becoming the aspect ratio method of choice in some European countries.

Zoom

In this technique, resolution is sacrificed in favor of filling the frame, since a significant percentage of signal lines are discarded. The cropping of the vertical dimension also makes this perhaps the most difficult technique to execute in the NLE.

(click thumbnail)

Difficulty arises from the fact that care must be taken not to discard portions of the picture containing vital information such as a soccer ball crossing the bottom of the screen, or an airplane near the top. Such positioning issues will be the norm rather than the exception with news material. Therefore, the conversion within the NLE must have some degree of controllability regarding the positioning of the crop. The examples below, calculated in terms of SD conversion at 625 lines for mathematical consistency, hold true proportionality at HD resolutions. They show possible crop positionings that should be included in the automatic conversion capabilities of the nonlinear editor, which should also offer a manual interface to adjust the crop on a line-by-line basis

(click thumbnail)

Multiple HD Resolution Formats

Just a few years ago, this subject provided the great divide between HD equipment vendors. There was a defined and adamant 1080i camp, and an equally vociferous bloc that insisted that only 720p could be considered “true” high definition. We now reap the benefits of that early dispute.

Probably because the original discussion was so strident and neither side shifted its position, this issue has been dealt with in manifold ways in products developed over the last decade. All of these characteristics should be embodied in any acceptable file based HD news production system. All are in existence in current products, in which the great debate has been reduced to a simple file attribute.

Acquisition Agility: Shoot 720p or 1080i on a shot-by-shot basis.

Ingest Agility: Record to file in 720p or 1080i on an event-by-event basis.

Editing Agility: Use 720p and 1080i material interchangeably on the NLE timeline.

Playout Agility: Play from 720p and 1080i source files interchangeably (including back-to-back) on the playout server.

Note that, even though a given operation may opt for one video format or the other as house standard, that is no guarantee that all material in a file-based storage system will adhere to that standard. File based syndication services may offer a mixed bag of 720p and 1080i, even within the context of a single service, and those files should be usable in the non-linear editor.

Multiple Codecs

There are a mind-boggling variety of methods in existence for producing very high-quality, highly compressed, high-definition moving images, and each has its adherents and detractors. A list that is almost certain to be obsolete as soon as it is read includes:

MPEG2, Intraframe and LGOP

DVCPro 100

JPEG2000

Canopus HQ

DNxHD

Sorenson (QuickTime)

MPEG4 AVC\H.264

Windows Media 9/ VC-1

Cineform (Adobe Premiere)

It’s worth noting that some of the codecs shown above are in fact “intermediate” codecs usually only encountered internally in some nonlinear editors. As such, they are optimized for editing. The use of such codecs in powerful software-only editors such as Canopus EDIUS, Apple FinalCut Pro, and Adobe Premiere, all of which target a much larger market than broadcast news, has led to the development of highly efficient software-only transcoding technology in which Moore’s Law plays an important role.

This efficient transcoding capability, the development of which was driven by the economics of the larger generalized editing market, has now made its way into broadcast nonlinear editors that also include features specific to working in large shared environments in which fast turnaround, deterministic throughput, and extremely high reliability are required. Transcoding is now routine and fast, and this ability drives a set of codec-related capabilities that should be considered requisite in the HD news production workflow.

Ingest Agility: File-based ingest must accept all widely used acquisition codecs, transcoding if needed.

Storage Agility: Multiple codec/file types must exist side-by-side in shared storage.

Editing Agility: NLE must be able to place all widely used acquisition/storage codecs on the timeline without user intervention, and have robust import capability for other codec types.

Output Agility: NLE must be able to output in a variety of broadcast codecs to support a range of playout server types. NLE must have export capability in non-broadcast (Web, mobile, etc.) appropriate codecs.

Multiple Wrapper Types

Although one would think that MXF was the solution to this issue, it actually adds some degree of complexity. Different manufacturers of camcorders, servers, and non-linear editors have adopted different MXF Operating Patterns. However, a properly architected, tapeless production system should make this complexity transparent to the user.

It is difficult to discern any specific advantage to be found by storing MXF wrapped material within the production environment, whereas many benefits accrue from a system in which storage consists of “codec ready” primitive elementary files. This does not imply in any way that MXF or other transfer/streaming wrapper types should not be supported. On the contrary, storing codec-ready files allows the system to support any and all wrapper types, wrapping and/or unwrapping as needed for any incoming or outgoing transfers.

This approach yields two benefits. First, it reduces dramatically the number of times a given CPU somewhere in the system must be tasked with dealing with the wrappers. If wrapped files are stored on the system, unwrapping must take place every time any file is accessed for any purpose, as opposed to the more efficient case described in the previous paragraph. Secondly, the parsers used to wrap or unwrap the material can extract or input metadata as needed from a system-wide asset management system. In fact, the need to extract any metadata payload in the wrapper is one of the strongest arguments against the concept of storing MXF. Unless the metadata is extracted and migrated to a database, any search would require opening each and every file on the system.

MXF is, or course, only part of the requirement. In order to ensure compatibility with the full spectrum of storage, operating system, and output destinations, IT-oriented wrapper types including .avi, QuickTime, etc. must also be available for import/export.

Multiple Metadata Standards

In this area, as with codec types, the issue is not one of a lack of standardization but rather one of too many standards. These standards are applied in three ways. First is the dictionary, referring to a standard which defines the meaning of various field names in a database. The second is the schema, defining the relationship of the various fields in the database to each other. The third is the database file type ranging from Key Length Value binary encoding in the DSM-1 specification of MXF, through IT-oriented architectures such as SQL, all the way to simple, human readable XML.

While it is tempting to embark a “one-size-fits-all” approach aimed at eliminating all but one of the various dictionaries, schemas, and file types – it is also futile. There are simply too many different end-user configurations possible to allow such reduction. The situation is compounded by the fact that the interface situation at any given site will not be static. New metadata sources will become available from time to time and the system must provide a means to integrate them without additional software engineering effort.

Fortunately, there is a simple comparison to familiar consumer software tools that provides a model for how to address this multiplicity. The problem of creating an integrated metadata system that spans all different types of metadata repositories can be dissolved by the simple measure of mapping fields. This is a process familiar to anyone who has ever had to import data from a spreadsheet into a database, or, even more to the point, from something like Palm Desktop to Microsoft Outlook.

The obvious upshot of this is that the system must include a mapping tool. Unfortunately, this is not quite as simple as the Palm-to-Outlook example cited above because of the expectation that new metadata sources of unpredictable characteristics will constantly be encountered. The same profusion of standards that creates the issue dictates that this mapping tool must by capable of being adapted to these new sources.

The solution is to be found in applying standard IT tools to the problem. The asset management portion of the news production system must provide a toolset based on such data interchange standards as Extensible Stylesheet Language Transformations (XSLT) for configurable XML exchanges of data and the Simple Object Access Protocol (SOAP) for .net and/or Web services data exchanges. This toolset need not (indeed probably should not) be usable by individuals with low or average computer skills. It should, however, be appropriate for the skills of IT staff that will be called on at infrequent intervals to update the data interchange maps at any given site.

Interoperability

The Overview section of this paper included the sentence, “The HD news production system must provide the same ease of integration and transparency of interoperability that was universal when the BNC was the only signal connector, analog composite was the signal type, and RS-422 provided the control layer.” Attaining this goal is dependent on manufacturers adopting, without caveat, the IT philosophy that interoperability benefits the entire industry, and thereby benefits all participants in the industry. This philosophy should manifest itself in the HD news production system in at least the following ways.

Network Transparency

There are certainly valid technical reasons for deploying proprietary technologies internally in any high performance audio/video storage, editing, and playout systems. Indeed, these technologies are what make possible scalable, reliable systems with the deterministic performance characteristics needed for real-time editing and playout. General purpose off-the-shelf technology does not do this, offering at best a statistical probability of delivering such performance some of the time.

But none of that dictates inaccessibility between such systems and other vendors’ products, nor should it require the purchase of network gateways, management software, or file translators. In this case, standard IT does offer appropriate solutions, since the extreme performance demands cited in the previous paragraph do not exist. No matter what the internal network or storage architecture of the storage/edit/playout system, it must be accessible… without additional cost… via the Common Internet File System (CIFS) over an open and upgradeable network layer, currently Gigabit Ethernet.

And example of such architecture is shown here in which the same hardware and processes that manage the system’s native operation and connectivity also provide transparent asynchronous access via CIFS.

(click thumbnail)

Transfer Transparency

With network transparency so easily available, the ability to move files from system to system should be equally transparent. For the most basic types of transfers, the system should have simple FTP capability. This is not to say, however, that there is no role for sophisticated transfer managers that allow queuing, prioritization, etc., but rather that such software should not be required as a prerequisite to any file interchange whatsoever.

Note that many of the earlier sections of this paper have direct relevance here. A rich set of wrapping/unwrapping capability, metadata mapping, and ubiquitous transcoding ability all enter into the picture, and are requirements in the definition of a competent system.

Summary

As the reader, you surely have noticed by now that this paper breaks no new ground. It exposes no stunning technological breakthroughs, and that is the point. The system characteristics described in these pages should be considered foundational principles for the development of any tapeless news production system in the era of high-definition news. The concepts and ideas professed in this paper are also offered to serve as a checklist for those evaluating such systems.

This paper was presented to SMPTE on October 20, 2006.