《VQEG的高清电视测试计划》由会员分享，可在线阅读，更多相关《VQEG的高清电视测试计划（40页珍藏版）》请在金锄头文库上搜索。
1、The VQEG HDTV Test PlanOutlinel lIntroduction: Setting the stagel lHDTV Source Scenes and HRCsl lTest methodologiesl lKey challengesl lProposed VQEG approachl lFinal thoughts2 2HD is no longer the future. It is the present!l l“NBC will upgrade the rest of the networks top-rated “NBC will upgrade the
2、 rest of the networks top-rated late-night lineup into HDTV in 2005”late-night lineup into HDTV in 2005”l l“The Fox network is better prepared then ever for its hi-“The Fox network is better prepared then ever for its hi-def broadcast of Super Bowl XXXIX.” def broadcast of Super Bowl XXXIX.” l l“The
3、 swanky future of hotels is a big-screen HDTV set.”“The swanky future of hotels is a big-screen HDTV set.”l l“2005 would be the year of high definition video and TV”“2005 would be the year of high definition video and TV”l l“Shipments of High Definition televisions and set-top “Shipments of High Def
4、inition televisions and set-top boxes are gaining momentum”boxes are gaining momentum”l l“HDTV content is increasing rapidly”“HDTV content is increasing rapidly”3 3HD studio workflows are mainstream l l“Adobe, HP, Intel, Microsoft, and Dell have joined “Adobe, HP, Intel, Microsoft, and Dell have joi
5、ned together to deliver a line of open, scalable, certified together to deliver a line of open, scalable, certified desktop HDV and HD solutions to deliver optimal desktop HDV and HD solutions to deliver optimal performance at a price point that is unbeatable today.” performance at a price point tha
6、t is unbeatable today.” l l“JVCs ProHD initiative represents an economical solution “JVCs ProHD initiative represents an economical solution to getting the stations into the HD production business. to getting the stations into the HD production business. The company has products to help with this tr
7、ansition on The company has products to help with this transition on two fronts.” two fronts.” 4 4Content acquisition is affordablel lPanasonic: The AG-HVX200 DVCPRO HD P2 represents the Panasonic: The AG-HVX200 DVCPRO HD P2 represents the companys first HD solid-state handheld camcorder. The AG-HVX
8、200 companys first HD solid-state handheld camcorder. The AG-HVX200 has two card slots and records in 1080/60i, 30p and 24p; 720/60p, has two card slots and records in 1080/60i, 30p and 24p; 720/60p, 30p and 24p; and DVCPRO50, DVCPRO or DV. It also records 480i 30p and 24p; and DVCPRO50, DVCPRO or D
9、V. It also records 480i and 480p on Mini-DV tape from a built-in drive. AG-HVX200 will and 480p on Mini-DV tape from a built-in drive. AG-HVX200 will come available in the fall for about $5,995. come available in the fall for about $5,995. l lAlec Shapiro, senior vice president of Sony Electronics B
10、roadcast Alec Shapiro, senior vice president of Sony Electronics Broadcast and Production Systems Division, touted the acceptance Sonys and Production Systems Division, touted the acceptance Sonys HDC-1000 1080P camera and the XDCAM tapeless acquisition HDC-1000 1080P camera and the XDCAM tapeless a
11、cquisition system have found in the industry. system have found in the industry. l lIn the high-end HD production arena, Shapiro noted the Sonys In the high-end HD production arena, Shapiro noted the Sonys HDCAM format is used by virtually 100 percent of the network HDCAM format is used by virtually
12、 100 percent of the network sitcoms produced in the United States.sitcoms produced in the United States. 5 5Viewer adoption is acceleratingl lMaxent (tm), announced the availability of its value-priced, high-Maxent (tm), announced the availability of its value-priced, high-performance 26-inch WXGA L
13、CD Display through Costco Wholesale stores performance 26-inch WXGA LCD Display through Costco Wholesale stores nationwide at a retail price of $1,199. nationwide at a retail price of $1,199. l lHDTVs keep getting more affordable. Olevias new 32-inch LCD HDTV has a HDTVs keep getting more affordable
14、. Olevias new 32-inch LCD HDTV has a lot of the right ingredients for a good low-cost HDTV.lot of the right ingredients for a good low-cost HDTV.l l10m homes around the world were watching HDTV programming on a high-10m homes around the world were watching HDTV programming on a high-definition TV se
15、t, according to research from market analysts In-Stat. definition TV set, according to research from market analysts In-Stat. l lBy the end of 2005, the worldwide total of these HDTV service households By the end of 2005, the worldwide total of these HDTV service households is projected to reach 15.
16、5m. By 2009, HDTV households worldwide are is projected to reach 15.5m. By 2009, HDTV households worldwide are forecast to reach 52m. forecast to reach 52m. l lHDTV services are currently widely available in five countries: Australia, HDTV services are currently widely available in five countries: A
17、ustralia, Canada, Japan, the United States, and South Korea. Canada, Japan, the United States, and South Korea. l lNearly half of all consumers plan to make their next television purchase a Nearly half of all consumers plan to make their next television purchase a high-definition (HDTV) television s
18、et, according to the CEA. high-definition (HDTV) television set, according to the CEA. 6 6Technology building blocks are in placel lBritish Sky Broadcasting (BSkyB) has selected Thomson as its British Sky Broadcasting (BSkyB) has selected Thomson as its primary HD decoder set top supplier as it prep
19、ares to roll out a new primary HD decoder set top supplier as it prepares to roll out a new digital HDTV service in 2006. The company will use H.264-based digital HDTV service in 2006. The company will use H.264-based advanced video coding for its HD service. advanced video coding for its HD service
20、. l lRoyal Philips Electronics today unveiled a High-Definition Multimedia Royal Philips Electronics today unveiled a High-Definition Multimedia Interface (HDMI ) receiver integrated circuit (IC), the Interface (HDMI ) receiver integrated circuit (IC), the TDA9975A, to meet consumer demands for supe
21、rior image quality in TDA9975A, to meet consumer demands for superior image quality in next-generation TVs. next-generation TVs. l lThe single cable digital multimedia solution has arrived. High-The single cable digital multimedia solution has arrived. High-Definition video and audio capability quie
22、tly rolled into the NZ home Definition video and audio capability quietly rolled into the NZ home entertainment market in the lead-up to Christmas, with the release entertainment market in the lead-up to Christmas, with the release of a number of DVD players, projectors and TV sets from various of a
23、 number of DVD players, projectors and TV sets from various manufacturers featuring the new High-Definition Multimedia manufacturers featuring the new High-Definition Multimedia Interface (HDMIInterface (HDMITMTM) connectivity standard.) connectivity standard.7 7Differentiation based on quality is t
24、he next stepl lOnHD.TV has generated international attention for our list of the 10 OnHD.TV has generated international attention for our list of the 10 celebrities who look better or worse in crystal-clear High-Definition celebrities who look better or worse in crystal-clear High-Definition TV. The
25、 list has been featured in such publications as TV. The list has been featured in such publications as The Chicago The Chicago TribuneTribune, , The National EnquirerThe National Enquirer, , The New YorkThe New York PostPost and TV shows and TV shows such as such as Inside EditionInside Edition and
26、and The Fox Report With Shepard SmithThe Fox Report With Shepard Smith. . l lDriving much of the companys growth is the worldwide move to HD Driving much of the companys growth is the worldwide move to HD production, and Thomson has products for the acquisition, editing production, and Thomson has p
27、roducts for the acquisition, editing and distribution of both SD and HD content. The top of the and distribution of both SD and HD content. The top of the companys line in terms of image quality is its Viper FilmStream companys line in terms of image quality is its Viper FilmStream digital cinema ca
28、mera.digital cinema camera.8 8Hence the need for standardized metricsl lTo verify and make credible claimsl lTo objectively validate the promised media experience9 9Outlinel lIntroduction: Setting the stagel lHDTV Source Scenes and HRCsl lTest methodologiesl lKey challengesl lProposed VQEG approachl
29、 lFinal thoughts1010HD landscape todayBroadcastHead-endSet-top BoxDTH SatelliteCable DistributionIP TV1111Content acquisitionl lQuality: Consumer ($6K), Professional ($50K), Studio ($110K) cameras in use.l lStorage: HDV, HDCAM, DVCPRO-HD decks are prevalent.l lFrame Rate: 24p, 29.97i, 30i, 59.94p, 6
30、0p.l lFrame Size: 1920 x 1080 (1080p), 1920 x 540 (1080i), 1280 x 720 (720p).1212Source Scenes - Copyrightl lCopyright IssuesCopyright Issuesl lPublic DomainPublic Domainl lShot by VQEG Members (put into public domain Shot by VQEG Members (put into public domain or approved for unlimited research us
31、e)or approved for unlimited research use)l lPurchased specifically for testing (e.g., VClips)Purchased specifically for testing (e.g., VClips)l lProposal: All scenes must have non-Proposal: All scenes must have non-expiring use clause for VQEG testing and be expiring use clause for VQEG testing and
32、be obtainable by any interested partyobtainable by any interested party1313Source Scenes - Qualityl lFormats?l lCamera/recording quality?l lUncompressed?l lMildly compressed?l lProfessional Consumer (e.g., Sony HVR-Z1U)?l lAllowing some percentage of lower grade material will greatly increase availa
33、ble sources1414Source Scenes - Characteristicsl lMotion?l lDetail?l lColor?l lSize of source scene pool?l lPercent unknown to proponents?1515HRC Characteristicsl lBit-rates?l lCoder/decoders?l lError conditions?l lPre/post processing?l lPercent unknown to proponents?1616Compressionl lTechnologies: M
34、PEG4-AVC, MPEG2, VC-1, DV compression.l lBit rate: 8 Mbps 25 Mbpsl lRate Control: Variable and constant.l lRate Conversion: 3:2 pull down.1717Transmissionl lTechnologies: Satellite, Cable, IPl lPacket loss: 0-25%1818Decode and display processingl l3:2 pull downl lImage enhancementl lDe-interlacingl
35、lScaling1919Display Conditionsl lShould we test many types of video displays or should we use a high quality “transparent” video display?l lWhat combinations of display types and video formats should be tested?2020Display technologiesl lBroadcast CRTl lRear projectionl lLCDl lPlasmal lProjectors2121
36、HRC vectors at a glancel lContent acquisition:Content acquisition:l lCapture qualityCapture qualityl lStorage formatStorage formatl lFrame rateFrame ratel lFrame sizeFrame sizel lCompression:Compression:l lCoding technologyCoding technologyl lBit rateBit ratel lRate controlRate controll lRate conver
37、sionRate conversionl lTransmission:Transmission:l lTechnologyTechnologyl lPacket lossPacket lossl lDecode and display Decode and display Processing:Processing:l l3:2 pull down3:2 pull downl lImage enhancementImage enhancementl lDe-interlacingDe-interlacingl lScalingScalingl lDisplay:Display:l lBroad
38、cast CRTBroadcast CRTl lRear ProjectionRear Projectionl lLCDLCDl lPlasmaPlasmal lProjectorsProjectors2222Outlinel lIntroduction: Setting the stagel lHDTV HRCsl lTest methodologiesl lKey challengesl lProposed VQEG approachl lFinal thoughts2323Test methodsl lSubjective testingl lSequence processing242
39、4Subjective Methodologyl lWhat subjective testing methodology should we use?l lDouble/single stimulus?Double/single stimulus?l lExplicit/hidden reference?Explicit/hidden reference?l lContinuous/discrete assessment?Continuous/discrete assessment?2525Subjective testingl lDouble Stimulus Impairment Sca
40、leDouble Stimulus Impairment Scalel l4 times picture height for all HD formats4 times picture height for all HD formatsl lDifferent display technologiesDifferent display technologiesl l24 subjects per test with:24 subjects per test with:l l20/20 visual acuity with or without eye glasses20/20 visual
41、acuity with or without eye glassesl lNormal color visionNormal color visionl lFamiliarity with the languageFamiliarity with the languagel lNo audioNo audiol lSubjects screened per BT.500Subjects screened per BT.500l lSubjects rejected if failed votes 1 in one Subjects rejected if failed votes 1 in o
42、ne session.session.2626Sequence processingl l1080p, 1080i and 720p formats.l lNo up-sampling or down-sampling.2727Outlinel lIntroduction: Setting the stagel lHDTV HRCsl lTest methodologiesl lKey challengesl lProposed VQEG approachl lFinal thoughts2828Key Challengesl lHD Contentl lReliable HD playbac
43、k for subjective testingl lInvestments required for HD testingl lTools for HRC error injectionl lReference display processing algorithmsl lSelection of back channel bandwidth for HD2929Outlinel lIntroduction: Setting the stagel lHDTV HRCsl lTest methodologiesl lKey challengesl lProposed VQEG approac
44、hl lFinal thoughts3030Step 1: Subjective testingl lSelect sequences and run a subjective test to get MOS results.l lInclude as many display technologies as possible.l lMake sure a variety of HRCs are used for the test sequences.l lCompile results from multiple labs to get a consolidated score for th
45、e sequences.3131Step 2: Evaluation of existing modelsl lRun through existing (already submitted) models through the same sequences.l lFollow procedure and rules for calibration in the test plan.l lInclude full reference, reduced reference and no reference models in the test.3232Step 3: Results analy
46、sis and next stepsl lAnalyze objective results to get a correlation to Analyze objective results to get a correlation to the subjective results.the subjective results.l lFollow evaluation metrics in the test plan.Follow evaluation metrics in the test plan.l lDecide on next steps based on the results
47、.Decide on next steps based on the results.l lIf correlations are high, we may have a model for HD If correlations are high, we may have a model for HD TV.TV.l lIf correlations are low, proponents will have a test set If correlations are low, proponents will have a test set to tweak their models.to
48、tweak their models.3333Lab readiness polll lOf the labs represented here:l lHow many of them have HD playback How many of them have HD playback capability?capability?l lOf those who dont have it now, how many of Of those who dont have it now, how many of them are planning to have this capability in
49、them are planning to have this capability in the next 3 - 6 months.the next 3 - 6 months.l lHow many labs are willing to run HD How many labs are willing to run HD subjective tests in the next 3 months?subjective tests in the next 3 months?3434Outlinel lIntroduction: Setting the stagel lHDTV HRCsl l
50、Test methodologiesl lKey challengesl lProposed VQEG approachl lFinal thoughts3535Final thoughtsl lHD TV metrics are in urgent need.l lWe should test existing models for their correlations before seeking new ones.l lHD testing is happening now in organizations. A timely standardized metric will have
51、the greatest impact.l lVQEG should complete testing the existing models in the next 6 months.3636Next stepsl lOff line meetings to coordinate HD testing activities.l lDate for next meeting?Date for next meeting?l lAction items captured so far:3737Calibration of model inputsl lPeak video level +/- 10
52、%Peak video level +/- 10%l lBlock level +/- 10%Block level +/- 10%l lHorizontal shift +/- 1 pixelHorizontal shift +/- 1 pixell lVertical shift +/- 1 lineVertical shift +/- 1 linel lHorizontal cropping 30 pixelsHorizontal cropping 30 pixelsl lVertical cropping 20 linesVertical cropping 20 linesl lNo
53、horizontal or vertical re-scalingNo horizontal or vertical re-scalingl lTemporal alignment +/- 1 frame excluding Temporal alignment +/- 1 frame excluding dropped/repeated framesdropped/repeated framesl lNo visible chroma differential timingNo visible chroma differential timingl lNo visible picture j
54、itterNo visible picture jitter3838Model evaluation metricsl lAttributes:l lPrediction accuracyPrediction accuracyl lPrediction monotonicityPrediction monotonicityl lPrediction consistencyPrediction consistencyl lMetrics:l lRoot mean square errorRoot mean square errorl lPearson linear correlationPearson linear correlationl lOutlier ratioOutlier ratio3939Results analysisl lILG is responsible4040
