1 <chapter id="multimedia">
2 <title>Wine and Multimedia</title>
5 This file contains information about the implementation of the
6 multimedia layer of Wine.
10 The implementation can be found in the dlls/winmm/ directory (and in
11 many of its subdirectories), but also in dlls/msacm/ (for the
12 audio compression/decompression manager) and dlls/msvideo/ (for the
13 video compression/decompression manager).
17 Written by &name-eric-pouech; <email>&email-eric-pouech;</email>
18 (Last updated: 02/16/2001)
21 <sect1 id="mm-overview">
22 <title>Overview</title>
25 The multimedia stuff is split into 3 layers. The low level (device
26 drivers), mid level (MCI commands) and high level abstraction layers.
27 The low level layer has also some helper DLLs (like the MSACM/MSACM32
28 and MSVIDEO/MSVFW32 pairs).
32 The low level layer may depend on current hardware and OS services
33 (like OSS on Unix). Mid level (MCI) and high level layers must be
34 written independently from the hardware and OS services.
38 There are two specific low level drivers (one for wave input/output,
39 another one for MIDI output only), whose role is:
42 <para>help choosing one low level driver between many</para>
46 add the possibility to convert streams (ie ADPCM => PCM)
51 add the possibility to filter a stream (adding echo, equalizer...
52 to a wave stream), or modify the instruments that have to be
60 All of those components are defined as DLLs (one by one).
66 <title>Low level layers</title>
69 Please note that native low level drivers are not currently supported
70 in Wine, because they either access hardware components or require
71 VxDs to be loaded; Wine does not correctly supports those two so far.
75 The following low level layers are implemented (as built-in DLLs):
79 <title>(Wave form) Audio</title>
82 MMSYSTEM and WINMM call the real low level audio driver using the
83 wodMessage/widMessage which handles the different requests.
87 <title>OSS implementation</title>
90 The low level audio driver is currently only implemented for the
91 OpenSoundSystem (OSS) as supplied in the Linux and FreeBSD kernels by
92 <ulink url="http://www.4front-tech.com/">4Front Technologies</ulink>.
93 The presence of this driver is checked by configure (depends on the
94 <sys/soundcard.h> file). Source code resides in
95 dlls/winmm/wineoss/audio.c.
99 The implementation contains all features commonly used, but has
100 several problems (see TODO list).
104 Note that some Wine specific flag has been added to the wodOpen function,
105 so that the dsound DLL can share the /dev/dsp access. Currently, this
106 only provides mutual exclusion for both DLLs. Future extension could add
107 a virtual mixer between the two output streams.
114 verify all functions for correctness
119 Add virtual mixer between wave-out and dsound interfaces.
128 <title>Other sub systems</title>
131 No other is available. Could think of Sun Audio, remote audio systems
132 (using X extensions, ...), ALSA, EsounD, ArTs...
143 MMSYSTEM and WINMM call the low level driver functions using the
144 midMessage and the modMessage functions.
148 <title>OSS driver</title>
151 The low level audio driver is currently only implemented for the
152 OpenSoundSystem (OSS) as supplied in the Linux and FreeBSD kernels by
153 <ulink url="http://www.4front-tech.com/">4Front Technologies</ulink>.
154 The presence of this driver is checked by configure (depends on the
155 <sys/soundcard.h> file, and also some specific defines because
156 MIDI is not supported on all OSes by OSS). Source code resides in
157 dlls/winmm/wineoss/midi.c
161 Both Midi in and Midi out are provided. The type of MIDI devices
162 supported is external MIDI port (requires an MIDI capable device -
163 keyboard...) and OPL/2 synthesis (the OPL/2 patches for all
164 instruments are in midiPatch.c).
171 use better instrument definition for OPL/2 (midiPatch.c) or use
172 existing instrument definition (from playmidi or kmid) with a
178 have a look at OPL/3 ?
183 implement asynchronous playback of MidiHdr
188 implement STREAM'ed MidiHdr (question: how shall we share the code
189 between the midiStream functions in MMSYSTEM/WINMM and the code
190 for the low level driver)
195 use a more accurate read mechanism than the one of snooping on
196 timers (like select on fd)
205 <title>Other sub systems</title>
208 Could support other MIDI implementation for other sub systems (any
213 Could also implement a software synthesizer, either inside Wine or
214 using using MIDI loop back devices in an external program (like
215 timidity). The only trouble is that timidity is GPL'ed...
226 MMSYSTEM and WINMM call the low level driver functions using the
231 <title>OSS implementation</title>
234 The current implementation uses the OpenSoundSystem mixer, and resides
235 in dlls/winmm/wineoss/mixer.c
242 implement notification mechanism when state of mixer's controls
252 <title>Other sub systems</title>
258 implement mixing low level drivers for other mixers (ALSA...)
272 The AUX low level driver is the predecessor of the mixer driver
273 (introduced in Win 95).
277 <title>OSS driver</title>
280 The implementation uses the OSS mixer API, and is incomplete.
287 verify the implementation
292 check with what is done in mixer
297 open question: shall we implement it on top of the low level mixer
310 <title>Wine OSS</title>
313 All the OSS dependent functions are stored into the WineOSS DLL. It still
314 lack a correct installation scheme (as any multimedia device under Windows),
315 so that all the correct keys are created in the registry. This requires
316 an advanced model since, for example, the number of wave out devices can
317 only be known on the destination system (depends on the sound card driven
318 by the OSS interface). A solution would be to install all the multimedia
319 drivers through the SETUPX DLL; this is not doable yet (the multimedia
320 extension to SETUPX isn't written yet).
326 <title>Joystick</title>
329 The API consists of the joy* functions found in dlls/winmm/joystick/joystick.c.
330 The implementation currently uses the Linux joystick device driver
331 API. It is lacking support for enhanced joysticks and has not been
339 better support of enhanced joysticks (Linux 2.2 interface is available)
344 support more joystick drivers (like the XInput extension)
349 should load joystick DLL as any other driver (instead of hardcoding)
350 the driver's name, and load it as any low lever driver.
359 <title>Wave mapper (msacm.drv)</title>
362 The Wave mapper device allows to load on-demand codecs in order to
363 perform software conversion for the types the actual low level driver
364 (hardware). Those codecs are provided through the standard ACM
369 <title>Built-in</title>
372 A first working implementation for wave out as been provided (wave in
373 exists, but doesn't allow conversion).
377 Wave mapper driver implementation can be found in dlls/winmm/wavemap/
378 directory. This driver heavily relies on MSACM and MSACM32 DLLs which
379 can be found in dlls/msacm and dlls/msacm32. Those DLLs load ACM
380 drivers which provide the conversion to PCM format (which is normally
381 supported by low level drivers). ADPCM, MP3... fit into the category
386 There is currently no built-in ACM driver in Wine, so you must use
387 native ones if you're looking for non PCM playback.
394 check for correctness and robustness
402 <title>Native</title>
405 Seems to work quite ok (using of course native MSACM/MSACM32 DLLs)
406 Some other testings report some issues while reading back the registry
415 <title>MIDI mapper</title>
418 Midi mapper allows to map each one of 16 MIDI channels to a specific
419 instrument on an installed sound card. This allows for example to
420 support different MIDI instrument definition (XM, GM...). It also
421 permits to output on a per channel basis to different MIDI renderers.
425 <title>Built-in</title>
428 A built-in MIDI mapper can be found in dlls/winmm/midimap/. It partly
429 provides the same functionnality as the Windows' one. It allows to pick up
430 destination channels (you can map a given channel to a specific playback
431 device channel (see the configuration bits for more details).
438 implement the Midi mapper features (instrument on the fly modification)
439 if it has to be done as under Windows, it required parsing the midi
440 configuration files (didn't find yet the specs)
449 <title>Native</title>
452 The native midimapper from Win 98 works, but it requires a bunch of
453 keys in the registry which are not part of the Wine source yet.
460 add native midimapper keys to the registry to let it run. This
461 will require proper multimedia driver installation routines.
474 <title>Mid level drivers (MCI)</title>
477 The mid level drivers are represented by some common API functions,
478 mostly mciSendCommand and mciSendString. See status in chapter 3 for
479 more information. Wine implements several MCI mid level drivers
480 (status is given for both built-in and native implementation):
483 <para>TODO: (apply to all built-in MCI drivers)
487 use MMSYSTEM multitasking caps instead of the home grown
495 <title>CDAUDIO</title>
498 <title>Built-in</title>
501 The currently best implementation is the MCI CDAUDIO driver that can
502 be found in dlls/winmm/mcicda/mcicda.c. The implementation is mostly
503 complete, there have been no reports of errors. It makes use of
504 misc/cdrom.c Wine internal cdrom interface.
505 This interface has been ported on Linux, FreeBSD and NetBSD. (Sun
506 should be similar, but are not implemented.)
510 A very small example of a cdplayer consists just of the line
511 mciSendString("play cdaudio",NULL,0,0);
518 add support for other cdaudio drivers (Solaris...)
523 add support for multiple cdaudio devices (plus a decent
524 configuration scheme)
529 The DLL is not cleanly separated from the NTDLL DLL. The CDROM
530 interface should be exported someway (or stored in a Wine only DLL)
539 <title>Native</title>
542 Native MCICDA works also correctly... It uses the MSCDEX traps (on int
543 2f). However, some commands (like seeking) seem to be broken.
551 <title>MCIWAVE</title>
554 <title>Built-in</title>
557 The implementation is rather complete and can be found in
558 dlls/winmm/mciwave/audio.c. It uses the low level audio API (although
559 not abstracted correctly).
566 The MCI_STATUS command is broken.
576 check for correctness
581 better use of asynchronous playback from low level
586 better implement non waiting command (without the MCI_WAIT flag).
595 <title>Native</title>
598 Native MCIWAVE works also correctly.
606 <title>MCISEQ (MIDI sequencer)</title>
609 <title>Built-in</title>
612 The implementation can be found in dlls/winmm/mciseq/mcimidi.c. Except
613 from the Record command, should be close to completion (except for non
614 blocking commands, as many MCI drivers).
621 implement it correctly
626 finish asynchronous commands (especially for reading/record)
631 better implement non waiting command (without the MCI_WAIT flag).
636 implement the recording features
645 <title>Native</title>
648 Native MCIMIDI has been working but is currently blocked by scheduling
649 issues (mmTaskXXX no longer work).
654 midiStreamPlay get from time to time an incorrect MidiHdr when
655 using the native MCI sequencer
666 <title>MCIANIM</title>
669 <title>Built-in</title>
672 The implementation is in dlls/winmm/mcianim/.
679 implement it, probably using xanim or something similar.
688 <title>Native</title>
691 Native MCIANIM is reported to work (but requires native video DLLs
692 also, even though the built-in video DLLs start to work correctly).
700 <title>MCIAVI</title>
703 <title>Built-in</title>
706 The implementation is in dlls/winmm/mcianim/. Basic features are present,
707 simple playing is available, even if lots remain to be done. It rather
708 heavily relies on MSVIDEO/MSVFW32 DLLs pair to work.
715 finish the implementation
720 fix the audio/video synchronisation issue
729 <title>Native</title>
732 Native MCIAVI is reported to work (but requires native video DLLs
733 also). Some files exhibit some deadlock issues anyway.
743 <title>High level layers</title>
746 The rest (basically the MMSYSTEM and WINMM DLLs entry points). It also
747 provides the skeleton for the core functionality for multimedia
748 rendering. Note that native MMSYSTEM and WINMM do not currently work
749 under Wine and there is no plan to support them (it would require to
750 also fully support VxD, which is not done yet).
751 Moreover, native DLLs require 16 bit MCI and low level drivers. Wine
752 implements them as 32 bit drivers.
753 MCI and low level drivers can either be 16 or 32 bit for Wine.
760 it seems that some program check what's installed in registry
761 against value returned by drivers. Wine is currently broken
762 regarding this point.
767 add clean-up mechanisms when process detaches from MM DLLs
772 prepare for the 16/32 big split
777 check thread-safeness for MMSYSTEM and WINMM entry points
782 unicode entry points are badly supported
789 <title>MCI skeleton</title>
792 Implementation of what is needed to load/unload MCI drivers, and to
793 pass correct information to them. This is implemented in
794 dlls/winmm/mci.c. The mciSendString function uses command strings,
795 which are translated into normal MCI commands as used by
796 mciSendCommand with the help of command tables. The API can be found
797 in dlls/winmm/mmsystem.c and dlls/winmm/mci.c. The functions there
798 (mciOpen,mciSysInfo) handle mid level driver allocation and calls. The
799 implementation is not complete.
803 MCI drivers are seen as regular Wine modules, and can be loaded (with
804 a correct load order between builtin, native, so), as any
805 other DLL. Please note, that MCI drivers module names must bear the
806 .drv extension to be correctly understood.
810 The list of available MCI drivers is obtained as follows:
811 1. key 'mci' in [option] section from .winerc (or wineconf)
812 mci=CDAUDIO:SEQUENCER gives the list of MCI drivers (names, in
813 uppercase only) to be used in Wine.
814 2. This list, when defined, supersedes the mci key in
815 c:\windows\system.ini
819 Note that native VIDEODISC crashes when the module is loaded, which
820 occurs when the MCI procedures are initialised. Make sure that this is
821 not in the list from above. Try adding:
822 mci=CDAUDIO:SEQUENCER:WAVEAUDIO:AVIVIDEO:MPEGVIDEO
823 to the [options] section of the wine config file.
830 correctly handle the MCI_ALL_DEVICE_ID in functions.
835 finish mapping 16 <=> 32 of MCI structures and commands
840 MCI_SOUND is not handled correctly (should not be sent to MCI
841 driver => same behavior as MCI_BREAK)
846 implement auto-open feature (ie, when a string command is issued
847 for a not yet opened device, MCI automatically opens it)
856 <title>MCI multi-tasking</title>
859 Multi-tasking capabilities used for the MCI drivers are provided in
860 dlls/winmm/mmsystem.c.
867 mmTaskXXX functions are currently broken because the 16 loader does
868 not support binary command lines => provide Wine's own mmtask.tsk not
869 using binary command line.
874 the Wine native MCI drivers should use the mmThreadXXX API (but since
875 all built-in MCI drivers are 32 bit, this would require a special
876 flag to mark 32 bit entry points)
885 <title>Timers</title>
888 It currently uses a service thread, run in the context of the calling
889 process, which should correctly mimic Windows behavior.
896 Check if minimal time is satisfactory for most programs.
901 current implementation may let a timer tick (once) after it has
914 The API consists of the mmio* functions found in dlls/winmm/mmio.c.
915 Seems to work ok in most of the cases. There's some linear/segmented
916 issues with 16 bit code. There are also some bugs when writting MMIO
923 <title>sndPlayXXX functions</title>
926 Seem to work correctly.
934 <title>Multimedia configuration</title>
937 Currently, multimedia configuration heavily relies on Win 3.x
942 <title>Drivers</title>
945 Since all multimedia drivers (MCI, low level ones, ACM drivers,
946 mappers) are, at first, drivers they need to appear in the [mci] or
947 [mci32] section of the system.ini file.
948 Since all drivers are, at first, DLLs, you can choose to load their
949 Wine's (built-in) or Windows (native) version.
958 A default [mci] section (in system.ini) looks like (see the note above
966 waveaudio=mciwave.drv
968 videodisc=mcipionr.drv
974 By default, the list of loadable MCI drivers will be made of those
975 drivers (in the [mci] section).
979 The list of loadable (recognized) MCI drivers can be altered in the
980 [option] section of the wine config file, like:
981 mci=CDAUDIO:SEQUENCER:WAVEAUDIO:AVIVIDEO:MPEGVIDEO
988 use a default registry setting to bypass this (ugly) configuration
998 <title>Low level drivers</title>
1001 Configuration of low level drivers is done with the Wine configuration file.
1002 Default keys are provided in winedefault.reg.
1006 The registry keys used here differ from the Windows' one. Using the Windows' one
1007 would require implementing something equivalent to a (real) driver installation.
1008 Even if this would be necessary in a few cases (mainly using MS native multimedia)
1009 modules, there's no real need so far (or it hasn't been run into yet).
1013 See the configuration part of the User's Guide for more details.
1019 <title>Midi mapper</title>
1022 The Midi mapper configuration is the same as on Windows 9x. Under the key
1024 HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Multimedia\MIDIMap
1026 if the 'UseScheme' value is not set, or is set to a null value, the midi
1027 mapper will always use the driver identified by the 'CurrentInstrument'
1028 value. Note: Wine (for simplicity while installing) allows to define
1029 'CurrentInstrument' as "#n" (where n is a number), whereas Windows only
1030 allows the real device name here. If UseScheme is set to a non null value,
1031 'CurrentScheme' defines the name of the scheme to map the different channels.
1032 All the schemes are available with keys like
1034 HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\MediaProperties\PrivateProperties\Midi\Schemes\%name_of_scheme%
1036 For every scheme, under this key, will be a sub-key (which name is usually
1037 a two digit index, starting at 00). Its default value is the name of the
1038 output driver, and the value 'Channels' lists all channels (of the 16
1039 standard MIDI ones) which have to be copied to this driver.
1043 To provide enhanced configuration and mapping capabilities, each driver
1044 can define under the key
1046 HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\MediaProperties\PrivateProperties\Midi\Ports\%driver_name%
1048 a link to and .IDF file which allows to remap channels internally (for
1049 example 9 -> 16), to change instruments identification, event
1050 controlers values. See the source file dlls/winmm/midimap/midimap.c
1051 for the details (this isn't implemented yet).
1059 To be done (use the same mechanism as MCI drivers configuration).
1068 To be done (use the same mechanism as MCI drivers configuration).
1075 <sect1 id="mm-arch">
1076 <title>Multimedia architecture</title>
1079 <title>Windows 9x multimedia architecture</title>
1083 Kernel space | Client applications
1086 | 16>| |<32 16>| |<32 16>| |<32 16>| |<32
1088 | +----|-----------|---------|------------|-------+
1089 | | | | | | | WinMM.dll
1090 | | | | | | | 32 bit
1091 | +----|-----------|---------|------------|-------+
1093 | +------+ | |<16 | | | |<16 |
1094 | | 16>| | | | | | | |
1096 | | +---------------+---+-------------+-------------+
1097 | | | waveInXXX | | mciXXX | *playSound* |
1098 | | | waveOutXXX | | | mmioXXX |
1099 | | | midiInXXX | | | timeXXX |
1100 | | | midiOutXXX | | | driverXXX |
1101 | | | midiStreamXXX | | | | MMSystem.dll
1102 | | | mixerXXX | | | | 16 bit
1103 +--------+ | | | auxXXX +---+ +---+ mmThread| |
1104 |MMDEVLDR|<------->| joyXXX | Call back | mmTask | |
1105 +--------+ | | +-----------+-----------+---------+-------------+
1107 | | | 16>| |<16>| 16>| |<16
1109 +--------+ | | +-------------+ +----------+
1110 | VxD |<------->| *.drv | | mci*.drv |
1111 +--------+ | | +--------------+ +-----------+
1112 | | | msacm.drv | | mciwave |
1113 | | +--------------+ +-----------+
1114 | | | midimap.drv | | mcimidi |
1115 | | +-------------+ +-----------+
1116 | | Low-level drivers | ... | MCI drivers
1120 | +-------------------------------+
1125 The important points to notice are:
1129 all drivers (and most of the core code) is 16 bit
1134 all hardware (or most of it) dependant code reside in the kernel
1135 space (which is not surprising)
1144 <title>Wine multimedia architecture</title>
1148 Kernel space | Client applications
1151 | 16>| |<32 16>| |<32 16>| |<32 16>| |<32
1153 | +------+ | | | | | | | |
1154 | |32/16>| | | | | | | | |
1155 | | v v v | | v v v v
1156 | | +---------------+---+-------------+-------------+
1157 | | | waveInXXX | | mciXXX | *playSound* |
1158 | | | waveOutXXX | | | mmioXXX | WinMM.dll
1159 | | | midiInXXX | | | timeXXX | 32 bit
1160 | | | midiOutXXX | | | driverXXX |
1161 | | | midiStreamXXX | | | | MMSystem.dll
1162 | | | mixerXXX | | | | 16 bit
1163 | | | auxXXX +---+ +---+ mmThread| |
1164 | | | joyXXX | Call back | mmTask | |
1165 | | +-----------+-----------+---------+-------------+
1167 | | 16>||<32 |<16>| 16>||<32>||<16
1169 +---------+ | | +-------------+ +----------+
1170 |HW driver|<------->| *.drv | | mci*.drv |
1171 +---------+ | | +--------------+ +-----------+
1172 | | | msacm.drv | | mciwave |
1173 | | +--------------+ +-----------+
1174 | | | midimap.drv | | mcimidi |
1175 | | +-------------+ +-----------+
1176 | | Low-level drivers | ... | MCI drivers
1180 | +-------------------------------+
1185 From the previous drawings, the most noticeable differences are:
1189 low-level drivers can either be 16 or 32 bit
1194 MCI drivers can either be 16 or 32 bit
1199 MMSystem and WinMM will be hosted in a single elfglue library
1204 no link between the MMSystem/WinMM pair on kernel space shall
1205 exist. For example, there will be a low level driver to talk to a
1206 UNIX OSS (Open Sound System) driver
1211 all built-in drivers (low-level and MCI) will be written as 32 bit
1217 all native drivers will be 16 bits drivers
1228 <title>MS ACM Dlls</title>
1231 <title>Contents</title>
1237 <title>Status</title>
1243 <title>Caching</title>
1246 The MSACM/MSACM32 keeps some data cached for all known ACM
1247 drivers. Under the key
1249 Software\Microsoft\AudioCompressionManager\DriverCache\<driver
1252 are kept for values:
1256 aFormatTagCache which contains an array of
1257 DWORD. There are two DWORDs per cFormatTags
1258 entry. The first DWORD contains a format tag
1259 value, and the second the associated maximum
1260 size for a WAVEFORMATEX structure.
1261 (Fields dwFormatTag and cbFormatSize from
1267 cFilterTags contains the number of tags supported by the driver
1273 cFormatTags contains the number of tags support
1274 by the driver for conversions.
1279 fdwSupport (the same as the one returned from
1287 The cFilterTags, cFormatTags, fdwSupport are the same
1288 values as the ones returned from acmDriverDetails
1297 <!-- Keep this comment at the end of the file
1300 sgml-parent-document:("wine-devel.sgml" "set" "book" "part" "chapter" "")