We report on VLA H I spectral line observations of five early-type galaxies classified as optically peculiar because of the presence of jets, ripples, or other optical fine structure. We detect H I within the primary beam (30' half-power beamwidth) in four of the five systems. However, in only one case is this gas associated with the targeted elliptical galaxy. In the other cases the H I is associated with a nearby gas-rich disk or dwarf galaxy. The one H I detection is for NGC 7626, where we tentatively detect an H I cloud lying between 20 and 40 kpc southwest of the galaxy center. Its origin is unclear. Our failure to detect obvious tidal H I features suggests that if these fine-structure elliptical galaxies are remnants of disk galaxy mergers, either the progenitors were gas-poor or they are well evolved and any gaseous tidal features have dispersed and/or been converted into other phases. Our targeted systems all reside in groups or clusters, and it seems likely that tidal H I is shorter lived in these environments than suggested by studies of more isolated merger remnants.