Anything that raises the temperature of the scrotum such as the use of hot tubs or long baths or over-tight clothing can harm sperm production. A variety of medicines and recreational drugs can also decrease male fertility. These include alcohol, cigarettes as well as certain medications. Studies have also shown that environmental factors have contributed to decreasing sperm counts over the years.
It's always recommended to take a second opinion with the experts.