1 Linux for S/390 and zSeries
3 Common Device Support (CDS)
4 Device Driver I/O Support Routines
9 Copyright, IBM Corp. 1999-2002
13 This document describes the common device support routines for Linux/390.
14 Different than other hardware architectures, ESA/390 has defined a unified
15 I/O access method. This gives relief to the device drivers as they don't
16 have to deal with different bus types, polling versus interrupt
17 processing, shared versus non-shared interrupt processing, DMA versus port
18 I/O (PIO), and other hardware features more. However, this implies that
19 either every single device driver needs to implement the hardware I/O
20 attachment functionality itself, or the operating system provides for a
21 unified method to access the hardware, providing all the functionality that
22 every single device driver would have to provide itself.
24 The document does not intend to explain the ESA/390 hardware architecture in
25 every detail.This information can be obtained from the ESA/390 Principles of
26 Operation manual (IBM Form. No. SA22-7201).
28 In order to build common device support for ESA/390 I/O interfaces, a
29 functional layer was introduced that provides generic I/O access methods to
32 The common device support layer comprises the I/O support routines defined
33 below. Some of them implement common Linux device driver interfaces, while
34 some of them are ESA/390 platform specific.
37 In order to write a driver for S/390, you also need to look into the interface
38 described in Documentation/s390/driver-model.txt.
40 Note for porting drivers from 2.4:
41 The major changes are:
42 * The functions use a ccw_device instead of an irq (subchannel).
43 * All drivers must define a ccw_driver (see driver-model.txt) and the associated
45 * request_irq() and free_irq() are no longer done by the driver.
46 * The oper_handler is (kindof) replaced by the probe() and set_online() functions
48 * The not_oper_handler is (kindof) replaced by the remove() and set_offline()
49 functions of the ccw_driver.
50 * The channel device layer is gone.
51 * The interrupt handlers must be adapted to use a ccw_device as argument.
52 Moreover, they don't return a devstat, but an irb.
53 * Before initiating an io, the options must be set via ccw_device_set_options().
56 read device characteristics
60 read configuration data.
63 get commands from extended sense data.
66 ccw_device_start_timeout()
67 ccw_device_start_key()
68 ccw_device_start_key_timeout()
69 initiate an I/O request.
72 resume channel program execution.
75 terminate the current I/O request processed on the device.
78 generic interrupt routine. This function is called by the interrupt entry
79 routine whenever an I/O interrupt is presented to the system. The do_IRQ()
80 routine determines the interrupt status and calls the device specific
81 interrupt handler according to the rules (flags) defined during I/O request
82 initiation with do_IO().
84 The next chapters describe the functions other than do_IRQ() in more details.
85 The do_IRQ() interface is not described, as it is called from the Linux/390
86 first level interrupt handler only and does not comprise a device driver
87 callable interface. Instead, the functional description of do_IO() also
88 describes the input to the device specific interrupt handler.
90 Note: All explanations apply also to the 64 bit architecture s390x.
93 Common Device Support (CDS) for Linux/390 Device Drivers
97 The following chapters describe the I/O related interface routines the
98 Linux/390 common device support (CDS) provides to allow for device specific
99 driver implementations on the IBM ESA/390 hardware platform. Those interfaces
100 intend to provide the functionality required by every device driver
101 implementaion to allow to drive a specific hardware device on the ESA/390
102 platform. Some of the interface routines are specific to Linux/390 and some
103 of them can be found on other Linux platforms implementations too.
104 Miscellaneous function prototypes, data declarations, and macro definitions
105 can be found in the architecture specific C header file
106 linux/include/asm-s390/irq.h.
108 Overview of CDS interface concepts
110 Different to other hardware platforms, the ESA/390 architecture doesn't define
111 interrupt lines managed by a specific interrupt controller and bus systems
112 that may or may not allow for shared interrupts, DMA processing, etc.. Instead,
113 the ESA/390 architecture has implemented a so called channel subsystem, that
114 provides a unified view of the devices physically attached to the systems.
115 Though the ESA/390 hardware platform knows about a huge variety of different
116 peripheral attachments like disk devices (aka. DASDs), tapes, communication
117 controllers, etc. they can all by accessed by a well defined access method and
118 they are presenting I/O completion a unified way : I/O interruptions. Every
119 single device is uniquely identified to the system by a so called subchannel,
120 where the ESA/390 architecture allows for 64k devices be attached.
122 Linux, however, was first built on the Intel PC architecture, with its two
123 cascaded 8259 programmable interrupt controllers (PICs), that allow for a
124 maximum of 15 different interrupt lines. All devices attached to such a system
125 share those 15 interrupt levels. Devices attached to the ISA bus system must
126 not share interrupt levels (aka. IRQs), as the ISA bus bases on edge triggered
127 interrupts. MCA, EISA, PCI and other bus systems base on level triggered
128 interrupts, and therewith allow for shared IRQs. However, if multiple devices
129 present their hardware status by the same (shared) IRQ, the operating system
130 has to call every single device driver registered on this IRQ in order to
131 determine the device driver owning the device that raised the interrupt.
133 In order not to introduce a new I/O concept to the common Linux code,
134 Linux/390 preserves the IRQ concept and semantically maps the ESA/390
135 subchannels to Linux as IRQs. This allows Linux/390 to support up to 64k
136 different IRQs, uniquely representing a single device each.
138 Up to kernel 2.4, Linux/390 used to provide interfaces via the IRQ (subchannel).
139 For internal use of the common I/O layer, these are still there. However,
140 device drivers should use the new calling interface via the ccw_device only.
142 During its startup the Linux/390 system checks for peripheral devices. Each
143 of those devices is uniquely defined by a so called subchannel by the ESA/390
144 channel subsystem. While the subchannel numbers are system generated, each
145 subchannel also takes a user defined attribute, the so called device number.
146 Both subchannel number and device number cannot exceed 65535. During driverfs
147 initialisation, the information about control unit type and device types that
148 imply specific I/O commands (channel command words - CCWs) in order to operate
149 the device are gathered. Device drivers can retrieve this set of hardware
150 information during their initialization step to recognize the devices they
151 support using the information saved in the struct ccw_device given to them.
152 This methods implies that Linux/390 doesn't require to probe for free (not
153 armed) interrupt request lines (IRQs) to drive its devices with. Where
154 applicable, the device drivers can use the read_dev_chars() to retrieve device
155 characteristics. This can be done without having to request device ownership
158 In order to allow for easy I/O initiation the CDS layer provides a
159 ccw_device_start() interface that takes a device specific channel program (one
160 or more CCWs) as input sets up the required architecture specific control blocks
161 and initiates an I/O request on behalf of the device driver. The
162 ccw_device_start() routine allows to specify whether it expects the CDS layer
163 to notify the device driver for every interrupt it observes, or with final status
164 only. See ccw_device_start() for more details. A device driver must never issue
165 ESA/390 I/O commands itself, but must use the Linux/390 CDS interfaces instead.
167 For long running I/O request to be canceled, the CDS layer provides the
168 ccw_device_halt() function. Some devices require to initially issue a HALT
169 SUBCHANNEL (HSCH) command without having pending I/O requests. This function is
170 also covered by ccw_device_halt().
173 read_dev_chars() - Read Device Characteristics
175 This routine returns the characteristics for the device specified.
177 The function is meant to be called with the device already enabled; that is,
178 at earliest during set_online() processing.
180 The ccw_device must not be locked prior to calling read_dev_chars().
182 The function may be called enabled or disabled.
184 int read_dev_chars(struct ccw_device *cdev, void **buffer, int length );
186 cdev - the ccw_device the information is requested for.
187 buffer - pointer to a buffer pointer. The buffer pointer itself
188 must contain a valid buffer area.
189 length - length of the buffer provided.
191 The read_dev_chars() function returns :
193 0 - successful completion
194 -ENODEV - cdev invalid
195 -EINVAL - an invalid parameter was detected, or the function was called early.
196 -EBUSY - an irrecoverable I/O error occurred or the device is not
200 read_conf_data(), read_conf_data_lpm() - Read Configuration Data
202 Retrieve the device dependent configuration data. Please have a look at your
203 device dependent I/O commands for the device specific layout of the node
204 descriptor elements. read_conf_data_lpm() will retrieve the configuration data
207 The function is meant to be called with the device already enabled; that is,
208 at earliest during set_online() processing.
210 The function may be called enabled or disabled, but the device must not be
213 int read_conf_data(struct ccw_device, void **buffer, int *length);
214 int read_conf_data_lpm(struct ccw_device, void **buffer, int *length, __u8 lpm);
216 cdev - the ccw_device the data is requested for.
217 buffer - Pointer to a buffer pointer. The read_conf_data() routine
218 will allocate a buffer and initialize the buffer pointer
219 accordingly. It's the device driver's responsibility to
220 release the kernel memory if no longer needed.
221 length - Length of the buffer allocated and retrieved.
222 lpm - Logical path mask to be used for retrieving the data. If
223 zero the data is retrieved on the next path available.
225 The read_conf_data() function returns :
226 0 - Successful completion
227 -ENODEV - cdev invalid.
228 -EINVAL - An invalid parameter was detected, or the function was called early.
229 -EIO - An irrecoverable I/O error occurred or the device is
231 -ENOMEM - The read_conf_data() routine couldn't obtain storage.
232 -EOPNOTSUPP - The device doesn't support the read configuration
236 get_ciw() - get command information word
238 This call enables a device driver to get information about supported commands
239 from the extended SenseID data.
242 ccw_device_get_ciw(struct ccw_device *cdev, __u32 cmd);
244 cdev - The ccw_device for which the command is to be retrieved.
245 cmd - The command type to be retrieved.
247 ccw_device_get_ciw() returns:
248 NULL - No extended data available, invalid device or command not found.
249 !NULL - The command requested.
252 ccw_device_start() - Initiate I/O Request
254 The ccw_device_start() routines is the I/O request front-end processor. All
255 device driver I/O requests must be issued using this routine. A device driver
256 must not issue ESA/390 I/O commands itself. Instead the ccw_device_start()
257 routine provides all interfaces required to drive arbitrary devices.
259 This description also covers the status information passed to the device
260 driver's interrupt handler as this is related to the rules (flags) defined
261 with the associated I/O request when calling ccw_device_start().
263 int ccw_device_start(struct ccw_device *cdev,
265 unsigned long intparm,
267 unsigned long flags);
268 int ccw_device_start_timeout(struct ccw_device *cdev,
270 unsigned long intparm,
274 int ccw_device_start_key(struct ccw_device *cdev,
276 unsigned long intparm,
279 unsigned long flags);
280 int ccw_device_start_key_timeout(struct ccw_device *cdev,
282 unsigned long intparm,
288 cdev : ccw_device the I/O is destined for
289 cpa : logical start address of channel program
290 user_intparm : user specific interrupt information; will be presented
291 back to the device driver's interrupt handler. Allows a
292 device driver to associate the interrupt with a
293 particular I/O request.
294 lpm : defines the channel path to be used for a specific I/O
295 request. A value of 0 will make cio use the opm.
296 key : the storage key to use for the I/O (useful for operating on a
297 storage with a storage key != default key)
298 flag : defines the action to be performed for I/O processing
299 expires : timeout value in jiffies. The common I/O layer will terminate
300 the running program after this and call the interrupt handler
301 with ERR_PTR(-ETIMEDOUT) as irb.
303 Possible flag values are :
305 DOIO_ALLOW_SUSPEND - channel program may become suspended
306 DOIO_DENY_PREFETCH - don't allow for CCW prefetch; usually
307 this implies the channel program might
309 DOIO_SUPPRESS_INTER - don't call the handler on intermediate status
311 The cpa parameter points to the first format 1 CCW of a channel program :
314 __u8 cmd_code;/* command code */
315 __u8 flags; /* flags, like IDA addressing, etc. */
316 __u16 count; /* byte count */
317 __u32 cda; /* data address */
318 } __attribute__ ((packed,aligned(8)));
320 with the following CCW flags values defined :
322 CCW_FLAG_DC - data chaining
323 CCW_FLAG_CC - command chaining
324 CCW_FLAG_SLI - suppress incorrect length
327 CCW_FLAG_IDA - indirect addressing
328 CCW_FLAG_SUSPEND - suspend
331 Via ccw_device_set_options(), the device driver may specify the following
332 options for the device:
334 DOIO_EARLY_NOTIFICATION - allow for early interrupt notification
335 DOIO_REPORT_ALL - report all interrupt conditions
338 The ccw_device_start() function returns :
340 0 - successful completion or request successfully initiated
341 -EBUSY - The device is currently processing a previous I/O request, or ther is
342 a status pending at the device.
343 -ENODEV - cdev is invalid, the device is not operational or the ccw_device is
346 When the I/O request completes, the CDS first level interrupt handler will
347 accumulate the status in a struct irb and then call the device interrupt handler.
348 The intparm field will contain the value the device driver has associated with a
349 particular I/O request. If a pending device status was recognized,
350 intparm will be set to 0 (zero). This may happen during I/O initiation or delayed
351 by an alert status notification. In any case this status is not related to the
352 current (last) I/O request. In case of a delayed status notification no special
353 interrupt will be presented to indicate I/O completion as the I/O request was
354 never started, even though ccw_device_start() returned with successful completion.
356 The irb may contain an error value, and the device driver should check for this
359 -ETIMEDOUT: the common I/O layer terminated the request after the specified
361 -EIO: the common I/O layer terminated the request due to an error state
363 If the concurrent sense flag in the extended status word in the irb is set, the
364 field irb->scsw.count describes the numer of device specific sense bytes
365 available in the extended control word irb->scsw.ecw[0]. No device sensing by
366 the device driver itself is required.
368 The device interrupt handler can use the following definitions to investigate
369 the primary unit check source coded in sense byte 0 :
372 SNS0_INTERVENTION_REQ 0x40
373 SNS0_BUS_OUT_CHECK 0x20
374 SNS0_EQUIPMENT_CHECK 0x10
377 SNS0_INCOMPL_DOMAIN 0x01
379 Depending on the device status, multiple of those values may be set together.
380 Please refer to the device specific documentation for details.
382 The irb->scsw.cstat field provides the (accumulated) subchannel status :
384 SCHN_STAT_PCI - program controlled interrupt
385 SCHN_STAT_INCORR_LEN - incorrect length
386 SCHN_STAT_PROG_CHECK - program check
387 SCHN_STAT_PROT_CHECK - protection check
388 SCHN_STAT_CHN_DATA_CHK - channel data check
389 SCHN_STAT_CHN_CTRL_CHK - channel control check
390 SCHN_STAT_INTF_CTRL_CHK - interface control check
391 SCHN_STAT_CHAIN_CHECK - chaining check
393 The irb->scsw.dstat field provides the (accumulated) device status :
395 DEV_STAT_ATTENTION - attention
396 DEV_STAT_STAT_MOD - status modifier
397 DEV_STAT_CU_END - control unit end
399 DEV_STAT_CHN_END - channel end
400 DEV_STAT_DEV_END - device end
401 DEV_STAT_UNIT_CHECK - unit check
402 DEV_STAT_UNIT_EXCEP - unit exception
404 Please see the ESA/390 Principles of Operation manual for details on the
405 individual flag meanings.
409 ccw_device_start() must be called disabled and with the ccw device lock held.
411 The device driver is allowed to issue the next ccw_device_start() call from
412 within its interrupt handler already. It is not required to schedule a
413 bottom-half, unless an non deterministically long running error recovery procedure
414 or similar needs to be scheduled. During I/O processing the Linux/390 generic
415 I/O device driver support has already obtained the IRQ lock, i.e. the handler
416 must not try to obtain it again when calling ccw_device_start() or we end in a
419 If a device driver relies on an I/O request to be completed prior to start the
420 next it can reduce I/O processing overhead by chaining a NoOp I/O command
421 CCW_CMD_NOOP to the end of the submitted CCW chain. This will force Channel-End
422 and Device-End status to be presented together, with a single interrupt.
423 However, this should be used with care as it implies the channel will remain
424 busy, not being able to process I/O requests for other devices on the same
425 channel. Therefore e.g. read commands should never use this technique, as the
426 result will be presented by a single interrupt anyway.
428 In order to minimize I/O overhead, a device driver should use the
429 DOIO_REPORT_ALL only if the device can report intermediate interrupt
430 information prior to device-end the device driver urgently relies on. In this
431 case all I/O interruptions are presented to the device driver until final
432 status is recognized.
434 If a device is able to recover from asynchronosly presented I/O errors, it can
435 perform overlapping I/O using the DOIO_EARLY_NOTIFICATION flag. While some
436 devices always report channel-end and device-end together, with a single
437 interrupt, others present primary status (channel-end) when the channel is
438 ready for the next I/O request and secondary status (device-end) when the data
439 transmission has been completed at the device.
441 Above flag allows to exploit this feature, e.g. for communication devices that
442 can handle lost data on the network to allow for enhanced I/O processing.
444 Unless the channel subsystem at any time presents a secondary status interrupt,
445 exploiting this feature will cause only primary status interrupts to be
446 presented to the device driver while overlapping I/O is performed. When a
447 secondary status without error (alert status) is presented, this indicates
448 successful completion for all overlapping ccw_device_start() requests that have
449 been issued since the last secondary (final) status.
451 Channel programs that intend to set the suspend flag on a channel command word
452 (CCW) must start the I/O operation with the DOIO_ALLOW_SUSPEND option or the
453 suspend flag will cause a channel program check. At the time the channel program
454 becomes suspended an intermediate interrupt will be generated by the channel
457 ccw_device_resume() - Resume Channel Program Execution
459 If a device driver chooses to suspend the current channel program execution by
460 setting the CCW suspend flag on a particular CCW, the channel program execution
461 is suspended. In order to resume channel program execution the CIO layer
462 provides the ccw_device_resume() routine.
464 int ccw_device_resume(struct ccw_device *cdev);
466 cdev - ccw_device the resume operation is requested for
468 The ccw_device_resume() function returns:
470 0 - suspended channel program is resumed
471 -EBUSY - status pending
472 -ENODEV - cdev invalid or not-operational subchannel
473 -EINVAL - resume function not applicable
474 -ENOTCONN - there is no I/O request pending for completion
477 Please have a look at the ccw_device_start() usage notes for more details on
478 suspended channel programs.
480 ccw_device_halt() - Halt I/O Request Processing
482 Sometimes a device driver might need a possibility to stop the processing of
483 a long-running channel program or the device might require to initially issue
484 a halt subchannel (HSCH) I/O command. For those purposes the ccw_device_halt()
487 ccw_device_halt() must be called disabled and with the ccw device lock held.
489 int ccw_device_halt(struct ccw_device *cdev,
490 unsigned long intparm);
492 cdev : ccw_device the halt operation is requested for
493 intparm : interruption parameter; value is only used if no I/O
494 is outstanding, otherwise the intparm associated with
495 the I/O request is returned
497 The ccw_device_halt() function returns :
499 0 - request successfully initiated
500 -EBUSY - the device is currently busy, or status pending.
501 -ENODEV - cdev invalid.
502 -EINVAL - The device is not operational or the ccw device is not online.
506 A device driver may write a never-ending channel program by writing a channel
507 program that at its end loops back to its beginning by means of a transfer in
508 channel (TIC) command (CCW_CMD_TIC). Usually this is performed by network
509 device drivers by setting the PCI CCW flag (CCW_FLAG_PCI). Once this CCW is
510 executed a program controlled interrupt (PCI) is generated. The device driver
511 can then perform an appropriate action. Prior to interrupt of an outstanding
512 read to a network device (with or without PCI flag) a ccw_device_halt()
513 is required to end the pending operation.
515 ccw_device_clear() - Terminage I/O Request Processing
517 In order to terminate all I/O processing at the subchannel, the clear subchannel
518 (CSCH) command is used. It can be issued via ccw_device_clear().
520 ccw_device_clear() must be called disabled and with the ccw device lock held.
522 int ccw_device_clear(struct ccw_device *cdev, unsigned long intparm);
524 cdev: ccw_device the clear operation is requested for
525 intparm: interruption parameter (see ccw_device_halt())
527 The ccw_device_clear() function returns:
529 0 - request successfully initiated
530 -ENODEV - cdev invalid
531 -EINVAL - The device is not operational or the ccw device is not online.
533 Miscellaneous Support Routines
535 This chapter describes various routines to be used in a Linux/390 device
536 driver programming environment.
540 Get the address of the device specific lock. This is then used in
541 spin_lock() / spin_unlock() calls.
544 __u8 ccw_device_get_path_mask(struct ccw_device *cdev);
546 Get the mask of the path currently available for cdev.