On operators in physical mathematics, part II, Proceedings of the Royal Society of London, Vol. 54, 1893, p. 121.
There is nothing by Heaviside in "Bartlett's Familiar Quotations," but the first phrase of Heaviside's aphorism, "Mathematics is an experimental science," is widely quoted. A web search can find it in dozens of places, but only one of the ones found by Google, for example, continues the quotation to the end of the sentence!*
Here is a larger piece of the section in which the saying appears: "In the preceding, I have purposely avoided giving any definition of 'equivalence.' Believing in example rather than precept, I have preferred to let the formulae, and the method of obtaining them, speak for themselves. Besides that, I could not give a satisfactory definition which I could feel sure would not require subsequent revision. Mathematics is an experimental science, and definitions do not come first, but later on. They make themselves, when the nature of the subject has developed itself. It would be absurd to lay down the law beforehand. Perhaps, therefore, the best thing I can do is to describe briefly several successive stages of knowledge related to equivalent and divergent series, being approximately representative of personal experience. (a). Complete ignorance. (b)...."
The epigraph in Operator Commutation Relations by Palle E. T. Jorgensen and Robert T. Moore, D. Reidel, Dordrecht / Boston / Lancaster, 1984, p. 56, incorporates the single sentence at the top of the page as a quotation. That epigraph is taken from a discussion of Heaviside's approach in The Historical Development of Quantum Theory, Vol. 3: The Formulation of Matrix Mechanics and Its Modifications, 1925-1926 by Jagdish Mehra and Helmut Rechenberg, Springer-Verlag, New York, 1982:
"Those who insisted on mathematical rigour, on clear definitions of the operators and well-defined equations obeyed by them could not take Heaviside's solution seriously. Against these objections Heaviside held that 'mathematics is an experimental science, and definitions do not come first, but later on.'"
It is on page 227 in vol. 3, and it is a statement by the authors Mehra and Rechenberg as they describe "the operator method" in connection with Max Born's work. This part of vol. 3 of Mehra-Rechenberg vol. 3 is primarily about Max Born, but Born is different from the other architects as he wrote lots of joint papers, he was the senior of them, and served as math teacher to Heisenberg, Jordan and other of the "younger" pioneers. And he was mentor to Heisenberg: At the time, Heisenberg asked Born if his pre-publication 1925 paper (Über die quantentheoretische Umdeutung kinematischer und mechanischer Beziehungen, Zeitschrift für Physik, vol. 33 (1925), no. 12, pp. 879-893) was worth publishing.** Heisenberg didn't even know what a matrix was before 1925. Born told him, and as it turned out, Heisenberg's paper won him the Nobel Prize.
This part of vol. 3 in the book set is really wonderful, i.e., V. 3 in vol. 3. Now vol. 3 is primarily about Born. There is a separate volume about Heisenberg. Born was the more mathematical of the architects of Quantum Mechanics, and he was mentor for and worked with Heisenberg. But Born was also greatly influenced by Dirac. Dirac had his own independent version of Heisenberg's paper, and realized more than perhaps some of the others the significance of non-commutativity. The operator method is thought to have started with Heaviside, but to have been revived by Dirac. And then others followed in Dirac's footsteps. Anyway 7 pages in V. 3 (vol. 3) are about the collaboration between Norbert Wiener and Max Born. Wiener was visiting Göttingen at the time. Mehra and Rehchenberg describe how, motivated by Dirac, Wiener had extended Heaviside's operator calculus, and how that was the start of the Wiener-Born collaboration, and how the operators slowly became to be accepted from then on, approved by the Demigod, David Hilbert, and then how, via John von Neumann, operators in Hilbert space became the current language of quantum mechanics. But Wiener's influence on Born, and on the operator formulation of Quantum Mechanics is generally underestimated, I think. Similarly, had it not been for Dirac, Wiener, Born and others, I think it is likely that the original work of Heaviside would have been completely forgotten.
to Quotes selected by Palle Jorgensen