Some skills apply a delay when used: while the delay is active neither that skill nor any other skill with a delay can be used. Nothing reduces the length of these delays but, since they begin at the action frame (the frame in the attack or casting animation in which the action is actually executed), increasing attack speed or cast rate will slightly reduce the interval between actions since the action frame will occur earlier in the animation: interval = action frame + delay.
The only exception is when skills with delays are cast by Chance to Cast items: multiple skills with delays can be cast.  However, while the skill delays are not applied, an 11 frame delay is applied from the action frame: during this delay no active skills can be used, although Scrolls of Identify and Town Portal placed in the belt can be used.  
This 11 frame delay does not interrupt an attack sequence if Chance to Cast is applied during that sequence. The delay can be applied from the action frame of any individual attack in the sequence, so if it's applied by an attack early in the sequence then the delay may end before that sequence ends. If multiple attacks in a sequence apply Chance to Cast, the delay is applied from the last attack in the sequence to apply it.