One can easily recognize his or her own cat by hearing his meow from beyond the wall, and can recall what he looks like without seeing them. As you can see from this example, object recognition is a multimodal process that involves the integration of multiple senses. However, over 40 years of research on object recognition has been devoted to understand visual processing of object recognition, and this trend has also been evident in studying the perirhinal cortex (PER), the core region for object recognition. To overcome this research gap, we developed a behavioral paradigm for rats to test multimodal object recognition, and then investigated how single neurons in the PER were activated during the process. In this paradigm, rats were required to maintain nose-poking to sample audiovisual cues simultaneously and then showed left or right choice response based on the identity of a given object. The rats showed a remarkable capability to recognize objects using only visual or auditory cues, even they learned the objects only in the multimodal situation, and this process required the PER. Single neurons in the PER exhibited object selective firing patterns, and further analysis revealed some neurons exhibited a preference for a specific modality (visual or auditory), while most neurons were invariant to modality information. These modality-invariant and modality-specific processes in the PER may enable both accurate object recognition and episodic memory recall.