Henry and Harriet LONG (formerly SHAW) 

found grand Aunt Harriet with my grandfather in 1851 which is great as it confirms that his mother was from Banbury and also her 2 sisters.....

see 1851 Thomas Sylvester Lankshear age 3 with Aunt Harriet LONG (formerly SHAW)

1851 Thomas Lankshane abt 1847 Clerkenwell, Middlesex Nephew
Harriet Long abt 1820 Banbury, Oxfordshire Wife
Henry Long abt 1812 Bermondsey, Surrey Head

HO107; Piece: 1561; Folio: 14; Page: 21 Bermondsey St Mary Magdalen

1816 ELIZABETH SHAW Christening and sister's Anne and Harriet

HARRIET SHAW - International Genealogical Index

HARRIET SHAW to HENRY JOHN LONG  Marriage: 04 SEP 1836 Holy Trinity, Newington, Surrey, England

1861 

RG9; Piece: 322; Folio: 73; Page: 52
Harriet Long abt 1820 Banbury, Oxfordshire Wife
Henry Long abt 1812 Bermondsey, Surrey Head

1841

Mary Long abt 1806 Surrey looks to be 55 not 35 so c1786 but ages rounded in 1841 census
Hancel (of course writing is Harriet)  Long abt 1821 Surrey - they just ticked Yes I suppose
Henry Long abt 1816 Surrey clearly Henry as a waiter 
James Long abt 1826 Surrey Tanner so hence connection with Shaw as John Shaw was a leather dresser

HO107; Piece 1047; Book: 7; Civil Parish: St Mary Magdalen Bermondsey; County: Surrey; Enumeration District: 13; Folio: 32; Page: 9

Deaths

Have looked at 1871 and 1881 and no sign of either Harriet or Henry

And looking 1861 to 1871
Possible deaths which fit well on age and area 
Deaths Dec 1866 (>99%) Long Harriet 47 Bermondsey 1d 74

Deaths Mar 1867 (>99%) Long Henry 59 Lambeth 1d 200
Deaths Mar 1869 (>99%) Long Henry John 57 Bermondsey 1d 83