Segfault using pp= and scale= to encode to mpeg4 File format: MJPG (Motion JPEG, from Zoran codec on a Matrox Marvel G400) Commandline: mencoder -ovc lavc -lavcopts vcodec=mpeg4 -nosound -vop \ scale=640:448,crop=696:488:20:44,pp=ci -o /dev/null \ cardiff1-segfault-sample.avi will cause a segmentation fault. Removing either pp=ci or scale=640:448 or both stops the segfault. (Other pp= options also give a segfault.) Putting format=yv12 right at the end of the -vop options, ie. as the _first_ 'filter' also prevents the segfault. See also the mplayer-users list archive, starting at: http://mplayerhq.hu/pipermail/mplayer-users/2003-February/030246.html Debugging information follows: Here's the mencoder -v output, taken from within gdb: Using GNU internationalization Original domain: messages Original dirname: /usr/share/locale Current domain: mplayer Current dirname: /usr/local/share/locale MEncoder dev-CVS-030226-17:28-3.2.2 (C) 2000-2003 Arpad Gereoffy (see DOCS) CPU: Advanced Micro Devices Athlon TB Thunderbird (Family: 6, Stepping: 4) Detected cache-line size is 64 bytes CPUflags: Type: 6 MMX: 1 MMX2: 1 3DNow: 1 3DNow2: 1 SSE: 0 SSE2: 0 Reading /root/.mplayer/codecs.conf: can't open '/root/.mplayer/codecs.conf': No such file or directory Reading /usr/local/etc/mplayer/codecs.conf: 50 audio & 135 video codecs File not found: 'frameno.avi' Reading config file /root/.mplayer/mencoder: No such file or directory init_freetype Using MMX (with tiny bit MMX2) Optimized OnScreenDisplay Not an URL! File size is 5242880 bytes success: format: 0 data: 0x0 - 0x500000 AVI file format detected. list_end=0x2284 ======= AVI Header ======= us/frame: 40000 (fps=25.000) max bytes/sec: 0 padding: 0 MainAVIHeader.dwFlags: (272) HAS_INDEX IS_INTERLEAVED frames total: 5205 initial: 0 streams: 2 Suggested BufferSize: 0 Size: 720 x 576 list_end=0x10F4 ==> Found video stream: 0 ======= STREAM Header ======= Type: vids FCC: MJPG (47504A4D) Flags: 0 Priority: 0 Language: 0 InitialFrames: 0 Rate: 1000000/40000 = 25.000 Start: 0 Len: 5205 Suggested BufferSize: 127189 Quality -1 Sample size: 0 found 'bih', 40 bytes of 40 ======= VIDEO Format ====== biSize 40 biWidth 720 biHeight 576 biPlanes 1 biBitCount 24 biCompression 1196444237='MJPG' biSizeImage 414720 =========================== list_end=0x2178 ==> Found audio stream: 1 ======= STREAM Header ======= Type: auds FCC: (0) Flags: 0 Priority: 0 Language: 0 InitialFrames: 1 Rate: 176400/4 = 44100.000 Start: 0 Len: 9181620 Suggested BufferSize: 88200 Quality -1 Sample size: 4 found 'wf', 16 bytes of 18 ======= WAVE Format ======= Format Tag: 1 (0x1) Channels: 2 Samplerate: 44100 avg byte/sec: 176400 Block align: 4 bits/sample: 16 cbSize: 0 list_end=0x2284 Broken chunk? chunksize=1396 (id=JUNK) list_end=0x22AD73F8 Found movie at 0x280C - 0x22AD73F8 stream_seek: WARNING! Can't seek to 0x22AD73F8 ! Auto-selected AVI video ID = 0 AVI video size=581740868 (5205) audio size=0 VIDEO: [MJPG] 720x576 24bpp 25.00 fps 22353.2 kbps (2728.7 kbyte/s) [V] filefmt:3 fourcc:0x47504A4D size:720x576 fps:25.00 ftime:=0.0400 Opening video filter: [expand=-1:-1:-1:-1:1] Expand: -1 x -1, -1 ; -1 (-1=autodetect) osd: 1 Opening video filter: [scale=640:448] SwScale params: 640 x 448 (-1=no scaling) Opening video filter: [crop=696:488:20:44] Crop: 696 x 488, 20 ; 44 Opening video filter: [pp=ci] [crop] query(Planar YV12) -> 1 [crop] query(Planar I420) -> 1 [crop] query(Planar IYUV) -> 1 [crop] query(Planar 444P) -> 1 [crop] query(Planar 422P) -> 1 [crop] query(Planar 411P) -> 1 ========================================================================== Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family INFO: libavcodec init OK! Selected video codec: [ffmjpeg] vfm:ffmpeg (FFmpeg MJPEG decoder) ========================================================================== Writing AVI header... [ffmpeg] aspect_ratio: 0.000000 VDec: vo config request - 720 x 576 (preferred csp: Planar 422P) [PP] Using external postprocessing filter, max q = 6. Trying filter chain: pp crop scale expand lavc VDec: using Planar 422P as output csp (no 1) Movie-Aspect is undefined - no prescaling applied. VO Config (720x576->720x576,flags=0,'MPlayer',0x50323234) REQ: flags=0x441 req=0x0 REQ: flags=0x441 req=0x400 SwScaler: reducing / aligning filtersize 6 -> 8 SwScaler: reducing / aligning filtersize 6 -> 8 SwScaler: reducing / aligning filtersize 6 -> 5 SwScaler: reducing / aligning filtersize 6 -> 5 SwScaler: BICUBIC scaler, from Planar YV12 to Planar YV12 using MMX2 SwScaler: using 8-tap MMX scaler for horizontal luminance scaling SwScaler: using 8-tap MMX scaler for horizontal chrominance scaling SwScaler: using n-tap MMX scaler for vertical scaling (YV12 like) SwScaler: 696x488 -> 640x448 REQ: flags=0x401 req=0x0 REQ: flags=0x401 req=0x0 videocodec: libavcodec (640x448 fourcc=58564944 [DIVX]) *** [pp] Exporting mp_image_t, 720x576x16bpp YUV planar, 829440 bytes *** [crop] Allocating mp_image_t, 720x576x12bpp YUV planar, 622080 bytes Program received signal SIGSEGV, Segmentation fault. The backtrace follows: #0 0x081f57fa in postProcess_MMX2 ( src=0x40707020 "ÿ}~}\200~{~\200\201\202\204\205\205\205\205\200\177~~}|{{z{{{||||\200\200\200\200\177\177\177\177\177~~~}}}|\200\200\177~~\177\200\201\201\201\201\201\200\177~}~~}}||||zz{}~\177\200\201\200\200\200\200\200\177~}~~~~}}}}{|~\177\201\201\201\201\202\202\201\200~}", '|' , "{{{{|}~\177\177~}}|||\201\202\203\204\205\206\207\207\203\202\200~|{{{{|}\177\177~|{||}}~~\177\177\201\202\202\202\202\202\203\203\204\204\205\205\205\204\203\203\202\203\204\204\204\204\203\202\201\201\201\201\202\202\202\203"..., srcStride=360, dst=0x40a25940 "ÿ~\200\200\202\200}\200\200\201\201\202\202\201\201\201~~~\177~~}}~}~~}\177~~\201\201\201\200\177~}}~~~\177~~}|\177~~}}~\177\200\177\177\177\177~~}}}~}}}}}}|||~}~\177\201\177\177\200\202\201\200\200\177\201\200\202\201\201\201\202\202\177\200\201\201\202\201\200\200\201\201\200\200\177\177\177\177~~~\200~~}}|{{||}}~~~}|~}\177\200\211\213\216\221\223\223\223\222\214\213\210\206\204\202\202\202\201\201\200\202\200~}{|z||\177\202\205\207\217\221\224\227\231\231\231\230\231\231\231\232\231\230\230\227\227\230\232\232\232\232\230\227\227\227\227\227\227\230\230\231"..., dstStride=360, width=360, height=576, QPs=0x8386bd0 '\001' , QPStride=0, isColor=1, c2=0x837cfc0) at libpostproc/postprocess_template.c:2593 #1 0x40a25940 in ?? () #2 0x081f6f54 in pp_postprocess (src=0x845f4b8, srcStride=0x845f4c8, dst=0x845f520, dstStride=0x845f530, width=360, height=576, QP_store=0x8386bd0 '\001' , QPStride=0, vm=0x8378fc8, vc=0x837cfc0, pict_type=360) at libpostproc/postprocess.c:476 #3 0x0809f3c6 in put_image (vf=0x8378df8, mpi=0x845f498) at vf_pp.c:127 ---Type to continue, or q to quit--- #4 0x0808f451 in decode_video (sh_video=0x835e978, start=0x407211c8 "ÿ\177\202\200\203\204\204\213\215\221\230\235\237¡¢¤¢££¤¥¥¦¦¦¦¦¦¦§§§¦¦¥¤¤¥¥¦¦§©ª¬ ­®®®¯¯°°°±¬«ª©ª« ®³µ¸º¼ÀÅÉÒÖÚÝÞßáãááààáâãããââááàààââââââââäãâãäãàÞÖÖÓÌû¸¸···¸¼ÀÅÇÎÒØÜÞßàâããããããââààááâããääããâááààâããâáßÝÜÝÝÝÝÝÜÜÜÜÚ×ÕÕÔÒÐÈý¸´± ª¤¤£¡\234\227\222\217"..., in_size=1081217480, drop_frame=0) at dec_video.c:327 #5 0x08063193 in main (argc=1081217480, argv=0x407211c8) at mencoder.c:1089 #6 0x4037cdc4 in __libc_start_main () from /lib/libc.so.6 Now the disassembly: Dump of assembler code from 0x81f57da to 0x81f581a: 0x81f57da : movq %mm1,(%edi,%ebx,1) 0x81f57de : movq (%esi,%ecx,2),%mm0 0x81f57e2 : movq (%eax,%ecx,2),%mm1 0x81f57e6 : movq %mm0,(%edi,%ebx,2) 0x81f57ea : movq %mm1,(%edx,%ebx,2) 0x81f57ee : movq (%esi,%ecx,4),%mm0 0x81f57f2 : movq (%eax,%ecx,4),%mm1 0x81f57f6 : movq %mm0,(%edi,%ebx,4) 0x81f57fa : movq %mm1,(%edx,%ebx,4) 0x81f57fe : lea (%eax,%ecx,4),%eax 0x81f5801 : lea (%edx,%ebx,4),%edx 0x81f5804 : movq (%eax,%ecx,1),%mm0 0x81f5808 : movq (%eax,%ecx,2),%mm1 0x81f580c : movq %mm0,(%edx,%ebx,1) 0x81f5810 : movq %mm1,(%edx,%ebx,2) 0x81f5814 : jmp 0x81f31ee 0x81f5819 : mov 0xfffffc90(%ebp),%eax End of assembler dump. And the register dump: eax 0x407211c8 1081217480 ecx 0x168 360 edx 0x40a3fae8 1084488424 ebx 0x168 360 esp 0xbfffde90 0xbfffde90 ebp 0xbfffe238 0xbfffe238 esi 0x40721060 1081217120 edi 0x40a3f980 1084488064 eip 0x81f57fa 0x81f57fa eflags 0x210246 2163270 cs 0x23 35 ss 0x2b 43 ds 0x2b 43 es 0x2b 43 fs 0x0 0 gs 0x7 7 st0 -nan(0x908885837f807eff) (raw 0xffff908885837f807eff) st1 -nan(0x8e87868580817efe) (raw 0xffff8e87868580817efe) st2 -nan(0x07c7b7b7a) (raw 0xffff000000007c7b7b7a) st3 -nan(0x07c7c7b7b) (raw 0xffff000000007c7c7b7b) st4 -nan(0x7c7b7c7b7c7b7c7c) (raw 0xffff7c7b7c7b7c7b7c7c) st5 -nan(0x838383838281807f) (raw 0xffff838383838281807f) st6 -nan(0x7f7f7e7d7d7e7e7f) (raw 0xffff7f7f7e7d7d7e7e7f) st7 -nan(0x101010101010101) (raw 0xffff0101010101010101) ---Type to continue, or q to quit--- fctrl 0x37f 895 fstat 0x120 288 ftag 0xaaaa 43690 fiseg 0x0 0 fioff 0x0 0 foseg 0x0 0 fooff 0x0 0 fop 0x0 0 xmm0 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = { 0x8000000000000000, 0x8000000000000000}, v16_int8 = { 0xff }, v8_int16 = {0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff}, v4_int32 = {0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff}, v2_int64 = {0xffffffffffffffff, 0xffffffffffffffff}, uint128 = 0xffffffffffffffffffffffffffffffff} xmm1 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = { 0x8000000000000000, 0x8000000000000000}, v16_int8 = { 0xff }, v8_int16 = {0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff}, v4_int32 = {0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff}, v2_int64 = {0xffffffffffffffff, 0xffffffffffffffff}, uint128 = 0xffffffffffffffffffffffffffffffff} xmm2 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = { 0x8000000000000000, 0x8000000000000000}, v16_int8 = { 0xff }, v8_int16 = {0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff}, v4_int32 = {0xffffffff, 0xffffffff, ---Type to continue, or q to quit--- 0xffffffff, 0xffffffff}, v2_int64 = {0xffffffffffffffff, 0xffffffffffffffff}, uint128 = 0xffffffffffffffffffffffffffffffff} xmm3 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = { 0x8000000000000000, 0x8000000000000000}, v16_int8 = { 0xff }, v8_int16 = {0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff}, v4_int32 = {0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff}, v2_int64 = {0xffffffffffffffff, 0xffffffffffffffff}, uint128 = 0xffffffffffffffffffffffffffffffff} xmm4 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = { 0x8000000000000000, 0x8000000000000000}, v16_int8 = { 0xff }, v8_int16 = {0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff}, v4_int32 = {0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff}, v2_int64 = {0xffffffffffffffff, 0xffffffffffffffff}, uint128 = 0xffffffffffffffffffffffffffffffff} xmm5 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = { 0x8000000000000000, 0x8000000000000000}, v16_int8 = { 0xff }, v8_int16 = {0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff}, v4_int32 = {0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff}, v2_int64 = {0xffffffffffffffff, 0xffffffffffffffff}, uint128 = 0xffffffffffffffffffffffffffffffff} xmm6 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = { 0x8000000000000000, 0x8000000000000000}, v16_int8 = { 0xff }, v8_int16 = {0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff}, v4_int32 = {0xffffffff, 0xffffffff, ---Type to continue, or q to quit--- 0xffffffff, 0xffffffff}, v2_int64 = {0xffffffffffffffff, 0xffffffffffffffff}, uint128 = 0xffffffffffffffffffffffffffffffff} xmm7 {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = { 0x8000000000000000, 0x8000000000000000}, v16_int8 = { 0xff }, v8_int16 = {0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff}, v4_int32 = {0xffffffff, 0xffffffff, 0xffffffff, 0xffffffff}, v2_int64 = {0xffffffffffffffff, 0xffffffffffffffff}, uint128 = 0xffffffffffffffffffffffffffffffff} mxcsr 0x1f80 8064 orig_eax 0xffffffff -1 mm0 {uint64 = 0x908885837f807eff, v2_int32 = {0x7f807eff, 0x90888583}, v4_int16 = {0x7eff, 0x7f80, 0x8583, 0x9088}, v8_int8 = {0xff, 0x7e, 0x80, 0x7f, 0x83, 0x85, 0x88, 0x90}} mm1 {uint64 = 0x8e87868580817efe, v2_int32 = {0x80817efe, 0x8e878685}, v4_int16 = {0x7efe, 0x8081, 0x8685, 0x8e87}, v8_int8 = {0xfe, 0x7e, 0x81, 0x80, 0x85, 0x86, 0x87, 0x8e}} mm2 {uint64 = 0x7c7b7b7a, v2_int32 = {0x7c7b7b7a, 0x0}, v4_int16 = { 0x7b7a, 0x7c7b, 0x0, 0x0}, v8_int8 = {0x7a, 0x7b, 0x7b, 0x7c, 0x0, 0x0, 0x0, 0x0}} mm3 {uint64 = 0x7c7c7b7b, v2_int32 = {0x7c7c7b7b, 0x0}, v4_int16 = { 0x7b7b, 0x7c7c, 0x0, 0x0}, v8_int8 = {0x7b, 0x7b, 0x7c, 0x7c, 0x0, 0x0, 0x0, 0x0}} mm4 {uint64 = 0x7c7b7c7b7c7b7c7c, v2_int32 = {0x7c7b7c7c, 0x7c7b7c7b}, v4_int16 = {0x7c7c, 0x7c7b, 0x7c7b, 0x7c7b}, v8_int8 = {0x7c, ---Type to continue, or q to quit--- 0x7c, 0x7b, 0x7c, 0x7b, 0x7c, 0x7b, 0x7c}} mm5 {uint64 = 0x838383838281807f, v2_int32 = {0x8281807f, 0x83838383}, v4_int16 = {0x807f, 0x8281, 0x8383, 0x8383}, v8_int8 = {0x7f, 0x80, 0x81, 0x82, 0x83, 0x83, 0x83, 0x83}} mm6 {uint64 = 0x7f7f7e7d7d7e7e7f, v2_int32 = {0x7d7e7e7f, 0x7f7f7e7d}, v4_int16 = {0x7e7f, 0x7d7e, 0x7e7d, 0x7f7f}, v8_int8 = {0x7f, 0x7e, 0x7e, 0x7d, 0x7d, 0x7e, 0x7f, 0x7f}} mm7 {uint64 = 0x101010101010101, v2_int32 = {0x1010101, 0x1010101}, v4_int16 = {0x101, 0x101, 0x101, 0x101}, v8_int8 = {0x1, 0x1, 0x1, 0x1, 0x1, 0x1, 0x1, 0x1}}