"786" is the total value of the letters of "Bismillah al-Rahman al-Rahim". In Arabic there are two methods of arranging letters. One method is the most common method known as the alphabetical method. Here we begin with Alif, ba, ta, tha etc. The other method is known as the Abjad method or ordinal method. In this method each letter has an arithmetic value assigned to it from one to one thousand. The letters are arranged in the following order: Abjad, Hawwaz, Hutti, Kalaman, Sa'fas, Qarshat, Sakhaz, Zazagh. This arrangement was done, most probably in the 3rd century of Hijrah during the 'Abbasid period, following other Semitic languages such as Phoenician, Aramaic, Hebrew, Syriac, Chaldean etc.
If you take the numeric values of all the letters of the Basmalah, according to the Abjad order, the total will be 786. In the Indian subcontinent the Abjad numerals became quite popular. Some people, mostly in India and Pakistan, use 786 as a substitute for Bismillah. They write this number to avoid writing the name of Allah or the Qur'anic ayah on ordinary papers. This tradition is not from the time of the Prophet -peace be upon him- or his Sahabah. It developed much later, perhaps during the later 'Abbasid period. We do not know of any reputable Imams or Jurists who used this number instead of the Bismillah.