This could be in one of two locations.

The modern-day World’s End is, as described by Wikipedia, here, “at the western end of King’s Road. It took its name from a public house.”

But then, the descriptions below from Edward Walford’s Old and New London: Volume 5, 1878, would suggest the place may have been further north. Unless the “Spring Garden” referred to stretched more than 2km from the north side of Lownde’s Square (here on a map), south-west to the modern World’s End?

…the commencement of the Knightsbridge Road is about fifty yards west of the Alexandra Hotel. Here, at the corner of the main road and of Wilton Place, stood formerly a tobacconist’s shop, which very much narrowed the thoroughfare, and was not removed till about the year 1840. It was occupied by an eccentric old woman, a Mrs. Dowell, who was so extremely partial to the Duke of Wellington, that she was constantly devising some new plan by which to show her regard for him. She sent him from time to time patties, cakes, and other delicacies of the like kind; and as it was found impossible to defeat the old woman’s pertinacity, the duke’s servants took in her presents. To such a pitch did she carry her mania, that she is said to have laid a knife and fork regularly for him at her own table day by day, constantly expecting that the duke would sooner or later do her the honour of dropping in and “taking pot luck” with her. In this hope, however, we believe we may safely assert that she was doomed to disappointment to the last.

At the back of the above-mentioned house was in former times one of the most noted suburban retreats in the neighbourhood of London, called the “Spring Garden,” a place of amusement formed in the grounds of an old mansion which stood on the north side of what is now Lowndes Square. Dr. King, of Oxford, mentions it in his diary as “an excellent spring garden;” and among the entries of the Virtuosi, or St. Luke’s Club, founded by Vandyck, is the following item:—”Paid—Spent at Spring Gardens, by Knightsbridge, forfeiture, £3 15s.” Pepys also, no doubt, refers to these same gardens in his “Diary,” when he writes:—”I lay in my drawers and stockings and waistcoat [at Kensington] till five of the clock, and so up; and being well pleased with our frolic, walked to Knightsbridge, and there ate a mess of cream; and so on to St. James’s.” Again, too, on another occasion:—”From the town, and away out of the Park, to Knightsbridge, and there ate and drank in the coach; and so home.” It is probable that the sign of the house in this Spring Garden was the “World’s End,” for the following entry in Mr. Pepys’ “Diary” can hardly refer to any other place but this:—”Forth to Hyde Park, but was too soon to go in; so went on to Knightsbridge, and there ate and drank at the ‘World’s End,’ where we had good things; and then back to the Park, and there till night, being fine weather, and much company.” And again, the very last entry in his “Diary,” under date of May 31st, 1669:—”To the Park, Mary Botelier and a Dutch gentleman, a friend of hers, being with us. Thence to the ‘World’s End,’ a drinking-house by the Park, and there merry, and so home late.”

The “World’s End,” it may be added, figures in a dialogue in Congreve’s Love for Love, in a way which implies that it bore no very high character.