Antibody phage display is regarded as a critical tool for the development of monoclonal antibodies for infectious diseases. The different classes of antibody libraries are classified based on the source of repertoire used to generate the libraries. Immune antibody libraries are generated from disease infected host or immunization against an infectious agent. Antibodies derived from immune libraries are distinct from those derived from naïve libraries as the host's in vivo immune mechanisms shape the antibody repertoire to yield high affinity antibodies. As the immune system is constantly evolving in accordance to the health state of an individual, immune libraries can offer more than just infection-specific antibodies but also antibodies derived from the memory B-cells much like naïve libraries. The combinatorial nature of the gene cloning process would give rise to a combination of natural and un-natural antibody gene pairings in the immune library. These factors have a profound impact on the coverage of immune antibody libraries to target both disease-specific and non-disease specific antigens. This review looks at the diverse nature of antibody responses for immune library generation and discusses the extended potential of a disease-specified immune library in the context of phage display.