I think that the adjective “empirical” is perhaps less problematic than Latour’s occasional appeals to empiricism. The noun “empiricism” is very ambiguous, between a notion of pure unconstructed experience (outside concepts and language games, and networks) and a sense of confronting concepts with trials and tests that are themselves quite constructed. Unfortunately Latour’s use of the notion of “second empiricism” oscillates between the two senses, which is why he can think AIME is “empirical” in a positive and self-conscious sense when others find him to be quite speculative.
I think that we need a way of situating the various possible empirical fixations or mono-interpretations that we may fall into. This is of a piece with the requirement that Feyerabend proposes in “How to be a Good Empiricist” of proliferation of alternatives as a way of ensuring empirical contact and of making conscious our empirical (which he later calls “cosmological”) presuppositions, so as to be able to vary them according to the needs of the situation. So empirical in the good sense combines two strands:
1) searching for and making expliciting the empirical presuppositions of various positions – this used to be called deconstruction, and Feyerabend calls it “cosmological criticism”.
2) using this deconstructive research not to eliminate but to introduce variation and plurality into your presuppositions, thus increasing theoretical resources rather than reducing them.
In that sense, as long as a thinker makes explicit in the terms of theirr problematic what they are doing, I see no problem with the idea of an “empirical metaphysics” and I think it has a fruitful ancestry (Feyerabend, Deleuze, Guattari, Lyotard, Foucault, Serres, and I would add Ernst Mach and Nietzsche).