Expand host names in Tramp's ad-hoc multi-hop file names

* doc/misc/tramp.texi (Quick Start Guide): Improve wording.
(Change file name syntax): Say, that `tramp-file-name-regexp' is
not constant.
(Ad-hoc multi-hops): Explain host name expansion.

* etc/NEWS: Mention that host names in Tramp ad-hoc multi-hop file
names must match the previous hop for methods like "su" or "sudo".
Fix typos.

* lisp/net/tramp.el (tramp-find-method, tramp-find-user):
Adapt docstring.
(tramp-find-host): Mark default value.
(tramp-dissect-file-name): Expand host name for hops.
(tramp-dissect-hop-name, tramp-make-tramp-hop-name): New defuns.
(tramp-clear-passwd): Simplify.

* test/lisp/net/tramp-tests.el (tramp-test02-file-name-dissect)
(tramp-test02-file-name-dissect-simplified)
(tramp-test02-file-name-dissect-separate)
(tramp-test26-file-name-completion): Extend tests.

doc/misc/tramp.texi

@@ -464,7 +464,7 @@ this case it is written as @code{host#port}.
 @cindex @option{plink} method
 
 If your local host runs an SSH client, and the remote host runs an SSH
-server, the most simple remote file name is
+server, the simplest remote file name is
 @file{@trampfn{ssh,user@@host,/path/to/file}}.  The remote file name
 @file{@trampfn{ssh,,}} opens a remote connection to yourself on the
 local host, and is taken often for testing @value{tramp}.
@@ -2459,9 +2459,10 @@ and @code{user@@} parts are optional.
 
 @defvar tramp-file-name-regexp
 This variable keeps a regexp which matches the selected remote file
-name syntax.  However, it is not recommended to use this variable in
+name syntax.  Its value changes after every call of
-external packages, a call of @code{file-remote-p} is much more
+@code{tramp-change-syntax}.  However, it is not recommended to use
-appropriate.
+this variable in external packages, a call of @code{file-remote-p} is
+much more appropriate.
 @ifinfo
 @pxref{Magic File Names, , , elisp}
 @end ifinfo
@@ -2585,9 +2586,9 @@ directory contents.
 @cindex multi-hop, ad-hoc
 @cindex proxy hosts, ad-hoc
 
-@value{tramp} file name syntax can accommodate ad hoc specification of
+@value{tramp} file name syntax can accommodate ad-hoc specification of
 multiple proxies without using @code{tramp-default-proxies-alist}
-configuration setup(@pxref{Multi-hops}).
+configuration setup (@pxref{Multi-hops}).
 
 Each proxy is specified using the same syntax as the remote host
 specification minus the file name part.  Each hop is separated by a
@@ -2600,8 +2601,6 @@ proxy @samp{bird@@bastion} to a remote file on @samp{you@@remotehost}:
 @kbd{C-x C-f @value{prefix}ssh@value{postfixhop}bird@@bastion|ssh@value{postfixhop}you@@remotehost@value{postfix}/path @key{RET}}
 @end example
 
-Proxies can take patterns @code{%h} or @code{%u}.
-
 @value{tramp} adds the ad-hoc definitions on the fly to
 @code{tramp-default-proxies-alist} and is available for re-use
 during that Emacs session.  Subsequent @value{tramp} connections to
@@ -2618,6 +2617,17 @@ For ad-hoc definitions to be saved automatically in
 @end lisp
 @end defopt
 
+Ad-hoc proxies can take patterns @code{%h} or @code{%u} like in
+@code{tramp-default-proxies-alist}.  The following file name expands
+to user @code{root} on host @code{remotehost}, starting with an
+@option{ssh} session on host @code{remotehost}:
+@samp{@value{prefix}ssh@value{postfixhop}%h|su@value{postfixhop}remotehost@value{postfix}}.
+
+On the other hand, if a trailing hop does not specifiy a host name,
+the host name of the previous hop is reused. Therefore, the following
+file name is equivalent to the previous example:
+@samp{@value{prefix}ssh@value{postfixhop}remotehost|su@value{postfixhop}@value{postfix}}.
+
 
 @node Remote processes
 @section Integration with other Emacs packages

etc/NEWS

@@ -172,7 +172,7 @@ the data.
 
 +++
 ** The Network Security Manager now allows more fine-grained control
-of what checks to run via the `network-security-protocol-checks'
+of what checks to run via the 'network-security-protocol-checks'
 variable.
 
 +++
@@ -356,7 +356,7 @@ shown in the currently selected window.
 ** Comint
 
 +++
-*** 'send-invisible' is now an obsolete alias for `comint-send-invisible'.
+*** 'send-invisible' is now an obsolete alias for 'comint-send-invisible'.
 Also, 'shell-strip-ctrl-m' is declared obsolete.
 
 +++
@@ -391,7 +391,7 @@ facilities to aid more casual SQL developers layout queries and
 complex expressions.
 
 *** 'sql-use-indent-support' (default t) enables SQL indention support.
-The `sql-indent' package from ELPA must be installed to get the
+The 'sql-indent' package from ELPA must be installed to get the
 indentation support in 'sql-mode' and 'sql-interactive-mode'.
 
 *** 'sql-mode-hook' and 'sql-interactive-mode-hook' changed.
@@ -420,7 +420,7 @@ This enables more efficient backends.  See the docstring of
 
 ** Package
 
-*** New function `package-get-version` lets packages query their own version.
+*** New function 'package-get-version' lets packages query their own version.
 Example use in auctex.el: (defconst auctex-version (package-get-version))
 
 *** New 'package-quickstart' feature.
@@ -747,6 +747,10 @@ are obsoleted in GVFS.
 *** The user option 'tramp-ignored-file-name-regexp' allows to disable
 Tramp for some look-alike remote file names.
 
++++
+*** For some connection methods, like "su" or "sudo", the host name in
+ad-hoc multi-hop file names must match the previous hop.
+
 ** Register
 ---
 *** The return value of method 'register-val-describe' includes the

lisp/net/tramp.el

@@ -1267,7 +1267,7 @@ entry does not exist, return nil."
 (defun tramp-find-method (method user host)
   "Return the right method string to use.
 This is METHOD, if non-nil. Otherwise, do a lookup in
-`tramp-default-method-alist'."
+`tramp-default-method-alist' and `tramp-default-method'."
   (when (and method
 	     (or (string-equal method "")
 		 (string-equal method tramp-default-method-marker)))
@@ -1292,7 +1292,7 @@ This is METHOD, if non-nil. Otherwise, do a lookup in
 (defun tramp-find-user (method user host)
   "Return the right user string to use.
 This is USER, if non-nil. Otherwise, do a lookup in
-`tramp-default-user-alist'."
+`tramp-default-user-alist' and `tramp-default-user'."
   (let ((result
 	 (or user
 	     (let ((choices tramp-default-user-alist)
@@ -1312,18 +1312,24 @@ This is USER, if non-nil. Otherwise, do a lookup in
 
 (defun tramp-find-host (method user host)
   "Return the right host string to use.
-This is HOST, if non-nil. Otherwise, it is `tramp-default-host'."
+This is HOST, if non-nil. Otherwise, do a lookup in
-  (or (and (> (length host) 0) host)
+`tramp-default-host-alist' and `tramp-default-host'."
-      (let ((choices tramp-default-host-alist)
+  (let ((result
-	    lhost item)
+	 (or (and (> (length host) 0) host)
-	(while choices
+	     (let ((choices tramp-default-host-alist)
-	  (setq item (pop choices))
+		   lhost item)
-	  (when (and (string-match (or (nth 0 item) "") (or method ""))
+	       (while choices
-		     (string-match (or (nth 1 item) "") (or user "")))
+		 (setq item (pop choices))
-	    (setq lhost (nth 2 item))
+		 (when (and (string-match (or (nth 0 item) "") (or method ""))
-	    (setq choices nil)))
+			    (string-match (or (nth 1 item) "") (or user "")))
-	lhost)
+		   (setq lhost (nth 2 item))
-      tramp-default-host))
+		   (setq choices nil)))
+	       lhost)
+	     tramp-default-host)))
+    ;; We must mark, whether a default value has been used.
+    (if (or (> (length host) 0) (null result))
+	result
+      (propertize result 'tramp-default t))))
 
 (defun tramp-dissect-file-name (name &optional nodefault)
   "Return a `tramp-file-name' structure of NAME, a remote file name.
@@ -1343,7 +1349,7 @@ default values are used."
 	    (host      (match-string (nth 3 tramp-file-name-structure) name))
 	    (localname (match-string (nth 4 tramp-file-name-structure) name))
 	    (hop       (match-string (nth 5 tramp-file-name-structure) name))
-	    domain port)
+	    domain port v)
 	(when user
 	  (when (string-match tramp-user-with-domain-regexp user)
 	    (setq domain (match-string 2 user)
@@ -1359,14 +1365,34 @@ default values are used."
 	    (setq host (replace-match "" nil t host))))
 
 	(unless nodefault
-	  (setq method (tramp-find-method method user host)
+	  (when hop
-		user (tramp-find-user method user host)
+	    (setq v (tramp-dissect-hop-name hop)
-		host (tramp-find-host method user host)))
+		  hop (and hop (tramp-make-tramp-hop-name v))))
+	  (let ((tramp-default-host
+		 (or (and v (not (string-match "%h" (tramp-file-name-host v)))
+			  (tramp-file-name-host v))
+		     tramp-default-host)))
+	    (setq method (tramp-find-method method user host)
+		  user (tramp-find-user method user host)
+		  host (tramp-find-host method user host)
+		  hop
+		  (and hop
+		       (format-spec hop (format-spec-make ?h host ?u user))))))
 
 	(make-tramp-file-name
 	 :method method :user user :domain domain :host host :port port
 	 :localname localname :hop hop)))))
 
+(defun tramp-dissect-hop-name (name &optional nodefault)
+  "Return a `tramp-file-name' structure of `hop' part of NAME.
+See `tramp-dissect-file-name' for details."
+  (tramp-dissect-file-name
+   (concat
+    tramp-prefix-format
+    (replace-regexp-in-string
+     (concat tramp-postfix-hop-regexp "$") tramp-postfix-host-format name))
+   nodefault))
+
 (defun tramp-buffer-name (vec)
   "A name for the connection buffer VEC."
   (let ((method (tramp-file-name-method vec))
@@ -1433,6 +1459,14 @@ the form (METHOD USER DOMAIN HOST PORT LOCALNAME &optional HOP)."
 	    tramp-postfix-host-format
 	    localname)))
 
+(defun tramp-make-tramp-hop-name (vec)
+  "Construct a Tramp hop name from VEC."
+  (replace-regexp-in-string
+   tramp-prefix-regexp ""
+   (replace-regexp-in-string
+    (concat tramp-postfix-host-regexp "$") tramp-postfix-hop-format
+    (tramp-make-tramp-file-name vec 'noloc))))
+
 (defun tramp-completion-make-tramp-file-name (method user host localname)
   "Construct a Tramp file name from METHOD, USER, HOST and LOCALNAME.
 It must not be a complete Tramp file name, but as long as there are
@@ -2313,7 +2347,7 @@ Falls back to normal file name handler if no Tramp file name handler exists."
 		       (tramp-message
 			v 1 "Interrupt received in operation %s"
 			(cons operation args)))
-		     ;; Propagate the quit signal.
+		     ;; Propagate the signal.
 		     (signal (car err) (cdr err)))
 
 		    ;; When we are in completion mode, some failed
@@ -4508,13 +4542,7 @@ Invokes `password-read' if available, `read-passwd' else."
 	(hop (tramp-file-name-hop vec)))
     (when hop
       ;; Clear also the passwords of the hops.
-      (tramp-clear-passwd
+      (tramp-clear-passwd (tramp-dissect-hop-name hop)))
-       (tramp-dissect-file-name
-	(concat
-	 tramp-prefix-format
-	 (replace-regexp-in-string
-	  (concat tramp-postfix-hop-regexp "$")
-	  tramp-postfix-host-format hop)))))
     (auth-source-forget
      `(:max 1 ,(and user-domain :user) ,user-domain
        :host ,host-port :port ,method))

test/lisp/net/tramp-tests.el

@@ -763,8 +763,8 @@ handled properly.  BODY shall not contain a timeout."
 	"|-:user2@host2"
 	"|-:user3@host3:/path/to/file"))
       (format "/%s:%s@%s|%s:%s@%s|%s:%s@%s:"
-	      "-" "user1" "host1"
+	      "method1" "user1" "host1"
-	      "-" "user2" "host2"
+	      "method2" "user2" "host2"
 	      "method3" "user3" "host3")))
 
     ;; Expand `tramp-default-user-alist'.
@@ -778,9 +778,9 @@ handled properly.  BODY shall not contain a timeout."
 	"/method1:host1"
 	"|method2:host2"
 	"|method3:host3:/path/to/file"))
-      (format "/%s:%s|%s:%s|%s:%s@%s:"
+      (format "/%s:%s@%s|%s:%s@%s|%s:%s@%s:"
-	      "method1" "host1"
+	      "method1" "user1" "host1"
-	      "method2" "host2"
+	      "method2" "user2" "host2"
 	      "method3" "user3" "host3")))
 
     ;; Expand `tramp-default-host-alist'.
@@ -794,9 +794,36 @@ handled properly.  BODY shall not contain a timeout."
 	"/method1:user1@"
 	"|method2:user2@"
 	"|method3:user3@:/path/to/file"))
-      (format "/%s:%s@|%s:%s@|%s:%s@%s:"
+      (format "/%s:%s@%s|%s:%s@%s|%s:%s@%s:"
-	      "method1" "user1"
+	      "method1" "user1" "host1"
-	      "method2" "user2"
+	      "method2" "user2" "host2"
+	      "method3" "user3" "host3")))
+
+    ;; Ad-hoc user name and host name expansion.
+    (setq tramp-default-method-alist nil
+	  tramp-default-user-alist nil
+	  tramp-default-host-alist nil)
+    (should
+     (string-equal
+      (file-remote-p
+       (concat
+	"/method1:user1@host1"
+	"|method2:user2@"
+	"|method3:user3@:/path/to/file"))
+      (format "/%s:%s@%s|%s:%s@%s|%s:%s@%s:"
+	      "method1" "user1" "host1"
+	      "method2" "user2" "host1"
+	      "method3" "user3" "host1")))
+    (should
+     (string-equal
+      (file-remote-p
+       (concat
+	"/method1:%u@%h"
+	"|method2:%u@%h"
+	"|method3:user3@host3:/path/to/file"))
+      (format "/%s:%s@%s|%s:%s@%s|%s:%s@%s:"
+	      "method1" "user3" "host3"
+	      "method2" "user3" "host3"
 	      "method3" "user3" "host3")))))
 
 (ert-deftest tramp-test02-file-name-dissect-simplified ()
@@ -1067,9 +1094,9 @@ handled properly.  BODY shall not contain a timeout."
 	      "/host1"
 	      "|host2"
 	      "|host3:/path/to/file"))
-	    (format "/%s|%s|%s@%s:"
+	    (format "/%s@%s|%s@%s|%s@%s:"
-		    "host1"
+		    "user1" "host1"
-		    "host2"
+		    "user2" "host2"
 		    "user3" "host3")))
 
 	  ;; Expand `tramp-default-host-alist'.
@@ -1083,9 +1110,35 @@ handled properly.  BODY shall not contain a timeout."
 	      "/user1@"
 	      "|user2@"
 	      "|user3@:/path/to/file"))
-	    (format "/%s@|%s@|%s@%s:"
+	    (format "/%s@%s|%s@%s|%s@%s:"
-		    "user1"
+		    "user1" "host1"
-		    "user2"
+		    "user2" "host2"
+		    "user3" "host3")))
+
+	  ;; Ad-hoc user name and host name expansion.
+	  (setq tramp-default-user-alist nil
+		tramp-default-host-alist nil)
+	  (should
+	   (string-equal
+	    (file-remote-p
+	     (concat
+	      "/user1@host1"
+	      "|user2@"
+	      "|user3@:/path/to/file"))
+	    (format "/%s@%s|%s@%s|%s@%s:"
+		    "user1" "host1"
+		    "user2" "host1"
+		    "user3" "host1")))
+	  (should
+	   (string-equal
+	    (file-remote-p
+	     (concat
+	      "/%u@%h"
+	      "|%u@%h"
+	      "|user3@host3:/path/to/file"))
+	    (format "/%s@%s|%s@%s|%s@%s:"
+		    "user3" "host3"
+		    "user3" "host3"
 		    "user3" "host3"))))
 
       ;; Exit.
@@ -1670,9 +1723,9 @@ handled properly.  BODY shall not contain a timeout."
 	      "/[/user1@host1"
 	      "|/user2@host2"
 	      "|/user3@host3]/path/to/file"))
-	    (format "/[/%s@%s|/%s@%s|%s/%s@%s]"
+	    (format "/[%s/%s@%s|%s/%s@%s|%s/%s@%s]"
-		    "user1" "host1"
+		    "method1" "user1" "host1"
-		    "user2" "host2"
+		    "method2" "user2" "host2"
 		    "method3" "user3" "host3")))
 
 	  ;; Expand `tramp-default-user-alist'.
@@ -1686,9 +1739,9 @@ handled properly.  BODY shall not contain a timeout."
 	      "/[method1/host1"
 	      "|method2/host2"
 	      "|method3/host3]/path/to/file"))
-	    (format "/[%s/%s|%s/%s|%s/%s@%s]"
+	    (format "/[%s/%s@%s|%s/%s@%s|%s/%s@%s]"
-		    "method1" "host1"
+		    "method1" "user1" "host1"
-		    "method2" "host2"
+		    "method2" "user2" "host2"
 		    "method3" "user3" "host3")))
 
 	  ;; Expand `tramp-default-host-alist'.
@@ -1702,9 +1755,36 @@ handled properly.  BODY shall not contain a timeout."
 	      "/[method1/user1@"
 	      "|method2/user2@"
 	      "|method3/user3@]/path/to/file"))
-	    (format "/[%s/%s@|%s/%s@|%s/%s@%s]"
+	    (format "/[%s/%s@%s|%s/%s@%s|%s/%s@%s]"
-		    "method1" "user1"
+		    "method1" "user1" "host1"
-		    "method2" "user2"
+		    "method2" "user2" "host2"
+		    "method3" "user3" "host3")))
+
+	  ;; Ad-hoc user name and host name expansion.
+	  (setq tramp-default-method-alist nil
+		tramp-default-user-alist nil
+		tramp-default-host-alist nil)
+	  (should
+	   (string-equal
+	    (file-remote-p
+	     (concat
+	      "/[method1/user1@host1"
+	      "|method2/user2@"
+	      "|method3/user3@]/path/to/file"))
+	    (format "/[%s/%s@%s|%s/%s@%s|%s/%s@%s]"
+		    "method1" "user1" "host1"
+		    "method2" "user2" "host1"
+		    "method3" "user3" "host1")))
+	  (should
+	   (string-equal
+	    (file-remote-p
+	     (concat
+	      "/[method1/%u@%h"
+	      "|method2/%u@%h"
+	      "|method3/user3@host3]/path/to/file"))
+	    (format "/[%s/%s@%s|%s/%s@%s|%s/%s@%s]"
+		    "method1" "user3" "host3"
+		    "method2" "user3" "host3"
 		    "method3" "user3" "host3"))))
 
       ;; Exit.
@@ -3491,6 +3571,7 @@ This tests also `make-symbolic-link', `file-truename' and `add-name-to-file'."
   (when (not (memq system-type '(cygwin windows-nt)))
     (let ((method (file-remote-p tramp-test-temporary-file-directory 'method))
 	  (host (file-remote-p tramp-test-temporary-file-directory 'host))
+	  (vec (tramp-dissect-file-name tramp-test-temporary-file-directory))
           (orig-syntax tramp-syntax))
       (when (and (stringp host) (string-match tramp-host-with-port-regexp host))
 	(setq host (match-string 1 host)))
@@ -3501,6 +3582,10 @@ This tests also `make-symbolic-link', `file-truename' and `add-name-to-file'."
 	       (if (tramp--test-expensive-test)
 		   (tramp-syntax-values) `(,orig-syntax)))
             (tramp-change-syntax syntax)
+	    ;; This has cleaned up all connection data, which are used
+	    ;; for completion.  We must refill the cache.
+	    (tramp-set-connection-property vec "property" nil)
+
             (let ;; This is needed for the `simplified' syntax.
                 ((method-marker
                   (if (zerop (length tramp-method-regexp))