_ _ ____ _ ___| | | | _ \| | / __| | | | |_) | | | (__| |_| | _ <| |___ \___|\___/|_| \_\_____| Changelog Version 7.61.1 (4 Sep 2018) Daniel Stenberg (4 Sep 2018) - THANKS: 7.61.1 status - RELEASE-NOTES: 7.61.1 - Curl_getoff_all_pipelines: ignore unused return values Since scan-build would warn on the dead "Dead store/Dead increment" Viktor Szakats (4 Sep 2018) - sftp: fix indentation Daniel Stenberg (4 Sep 2018) - [Przemysław Tomaszewski brought this change] sftp: don't send post-qoute sequence when retrying a connection Fixes #2939 Closes #2940 Kamil Dudka (3 Sep 2018) - url, vtls: make CURLOPT{,_PROXY}_TLS13_CIPHERS work This is a follow-up to PR #2607 and PR #2926. Closes #2936 Daniel Stenberg (3 Sep 2018) - [Jay Satiro brought this change] tool_operate: Add http code 408 to transient list for --retry - Treat 408 request timeout as transient so that curl will retry the request if --retry was used. Closes #2925 - [Jay Satiro brought this change] openssl: Fix setting TLS 1.3 cipher suites The flag indicating TLS 1.3 cipher support in the OpenSSL backend was missing. Bug: https://github.com/curl/curl/pull/2607#issuecomment-417283187 Reported-by: Kamil Dudka Closes #2926 - Curl_ntlm_core_mk_nt_hash: return error on too long password ... since it would cause an integer overflow if longer than (max size_t / 2). This is CVE-2018-14618 Bug: https://curl.haxx.se/docs/CVE-2018-14618.html Closes #2756 Reported-by: Zhaoyang Wu - [Rikard Falkeborn brought this change] http2: Use correct format identifier for stream_id Closes #2928 Marcel Raad (2 Sep 2018) - test1148: fix precheck output "precheck command error" is not very helpful. Daniel Stenberg (1 Sep 2018) - all: s/int/size_t cleanup Assisted-by: Rikard Falkeborn Closes #2922 - ssh-libssh: use FALLTHROUGH to silence gcc8 Jay Satiro (31 Aug 2018) - tool_operate: Fix setting proxy TLS 1.3 ciphers Daniel Stenberg (31 Aug 2018) - [Daniel Gustafsson brought this change] cookies: support creation-time attribute for cookies According to RFC6265 section 5.4, cookies with equal path lengths SHOULD be sorted by creation-time (earlier first). This adds a creation-time record to the cookie struct in order to make cookie sorting more deterministic. The creation-time is defined as the order of the cookies in the jar, the first cookie read fro the jar being the oldest. The creation-time is thus not serialized into the jar. Also remove the strcmp() matching in the sorting as there is no lexicographic ordering in RFC6265. Existing tests are updated to match. Closes #2524 Marcel Raad (31 Aug 2018) - Don't use Windows path %PWD for SSH tests All these tests failed on Windows because something like sftp://%HOSTIP:%SSHPORT%PWD/ expanded to sftp://127.0.0.1:1234c:/msys64/home/bla/curl and then curl complained about the port number ending with a letter. Use the original POSIX path instead of the Windows path created in checksystem to fix this. Closes https://github.com/curl/curl/pull/2920 Jay Satiro (29 Aug 2018) - CURLOPT_SSL_CTX_FUNCTION.3: clarify connection reuse warning Reported-by: Daniel Stenberg Closes https://github.com/curl/curl/issues/2916 Daniel Stenberg (28 Aug 2018) - THANKS-filter: dedup Daniel Jeliński - RELEASE-NOTES: synced - CURLOPT_ACCEPT_ENCODING.3: list them comma-separated [ci skip] - CURLOPT_SSL_CTX_FUNCTION.3: might cause unintended connection reuse [ci skip] Added a warning! Closes #2915 - curl: fix time-of-check, time-of-use race in dir creation Patch-by: Jay Satiro Detected by Coverity Fixes #2739 Closes #2912 - cmdline-opts/page-footer: fix edit mistake There was a missing newline. follow-up to a7ba60bb7250 - docs: clarify NO_PROXY env variable functionality Reported-by: Kirill Marchuk Fixes #2773 Closes #2911 Marcel Raad (24 Aug 2018) - lib1522: fix curl_easy_setopt argument type CURLOPT_POSTFIELDSIZE is a long option. - curl_threads: silence bad-function-cast warning As uintptr_t and HANDLE are always the same size, this warning is harmless. Just silence it using an intermediate uintptr_t variable. Closes https://github.com/curl/curl/pull/2908 Daniel Stenberg (24 Aug 2018) - README: add appveyor build badge [ci skip] Closes #2913 - [Ihor Karpenko brought this change] schannel: client certificate store opening fix 1) Using CERT_STORE_OPEN_EXISTING_FLAG ( or CERT_STORE_READONLY_FLAG ) while opening certificate store would be sufficient in this scenario and less-demanding in sense of required user credentials ( for example, IIS_IUSRS will get "Access Denied" 0x05 error for existing CertOpenStore call without any of flags mentioned above ), 2) as 'cert_store_name' is a DWORD, attempt to format its value like a string ( in "Failed to open cert store" error message ) will throw null pointer exception 3) adding GetLastError(), in my opinion, will make error message more useful. Bug: https://curl.haxx.se/mail/lib-2018-08/0198.html Closes #2909 - [Leonardo Taccari brought this change] gopher: Do not translate `?' to `%09' Since GOPHER support was added in curl `?' character was automatically translated to `%09' (`\t'). However, this behaviour does not seems documented in RFC 4266 and for search selectors it is documented to directly use `%09' in the URL. Apart that several gopher servers in the current gopherspace have CGI support where `?' is used as part of the selector and translating it to `%09' often leads to surprising results. Closes #2910 Marcel Raad (23 Aug 2018) - cookie tests: treat files as text Fixes test failures because of wrong line endings on Windows. Daniel Stenberg (23 Aug 2018) - libcurl-thread.3: expand somewhat on the NO_SIGNAL motivation Multi-threaded applictions basically MUST set CURLOPT_NO_SIGNAL to 1L to avoid the risk of getting a SIGPIPE. Either way, a multi-threaded application that uses libcurl/openssl needs to have a signhandler for or ignore SIGPIPE on its own. Based on discussions in #2800 Closes #2904 - RELEASE-NOTES: synced Marcel Raad (22 Aug 2018) - Tests: fixes for Windows - test 1268 requires unix sockets - test 2072 must be disabled also for MSYS/MinGW Daniel Stenberg (22 Aug 2018) - http2: abort the send_callback if not setup yet When Curl_http2_done() gets called before the http2 data is setup all the way, we cannot send anything and this should just return an error. Detected by OSS-Fuzz Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=10012 - http2: remove four unused nghttp2 callbacks Closes #2903 - x509asn1: use FALLTHROUGH ... as no other comments are accepted since 014ed7c22f51463 Marcel Raad (21 Aug 2018) - test1148: disable if decimal separator is not point Modifying the locale with environment variables doesn't work for native Windows applications. Just disable the test in this case if the decimal separator is something different than a point. Use a precheck with a small C program to achieve that. Closes https://github.com/curl/curl/pull/2786 - Enable more GCC warnings This enables the following additional warnings: -Wold-style-definition -Warray-bounds=2 instead of the default 1 -Wformat=2, but only for GCC 4.8+ as Wno-format-nonliteral is not respected for older versions -Wunused-const-variable, which enables level 2 instead of the default 1 -Warray-bounds also in debug mode through -ftree-vrp -Wnull-dereference also in debug mode through -fdelete-null-pointer-checks Closes https://github.com/curl/curl/pull/2747 - curl-compilers: enable -Wimplicit-fallthrough=4 for GCC This enables level 4 instead of the default level 3, which of the currently used comments only allows /* FALLTHROUGH */ to silence the warning. Closes https://github.com/curl/curl/pull/2747 - curl-compilers: enable -Wbad-function-cast on GCC This warning used to be enabled only for clang as it's a bit stricter on GCC. Silence the remaining occurrences and enable it on GCC too. Closes https://github.com/curl/curl/pull/2747 - configure: conditionally enable pedantic-errors Enable pedantic-errors for GCC >= 5 with --enable-werror. Before GCC 5, pedantic-errors was synonymous to -Werror=pedantic [0], which is still the case for clang [1]. With GCC 5, it became complementary [2]. Also fix a resulting error in acinclude.m4 as main's return type was missing, which is illegal in C99. [0] https://gcc.gnu.org/onlinedocs/gcc-4.9.0/gcc/Warning-Options.html [1] https://clang.llvm.org/docs/UsersManual.html#options-to-control-error-and-warning-messages [2] https://gcc.gnu.org/onlinedocs/gcc-5.1.0/gcc/Warning-Options.html Closes https://github.com/curl/curl/pull/2747 - Remove unused definitions Closes https://github.com/curl/curl/pull/2747 Daniel Stenberg (21 Aug 2018) - x509asn1: make several functions static and remove the private SIZE_T_MAX define and use the generic one. Closes #2902 - INTERNALS: require GnuTLS >= 2.11.3 Since the public pinning support was brought in e644866caf4. GnuTLS 2.11.3 was released in October 2010. Figured out in #2890 - http2: avoid set_stream_user_data() before stream is assigned ... before the stream is started, we have it set to -1. Fixes #2894 Closes #2898 - SSLCERTS: improve the openssl command line ... for extracting certs from a live HTTPS server to make a cacerts.pem from them. - docs/SECURITY-PROCESS: now we name the files after the CVE id - RELEASE-NOTES: synced - upload: change default UPLOAD_BUFSIZE to 64KB To make uploads significantly faster in some circumstances. Part 2 of #2888 Closes #2892 - upload: allocate upload buffer on-demand Saves 16KB on the easy handle for operations that don't need that buffer. Part 1 of #2888 - [Laurent Bonnans brought this change] vtls: reinstantiate engine on duplicated handles Handles created with curl_easy_duphandle do not use the SSL engine set up in the original handle. This fixes the issue by storing the engine name in the internal url state and setting the engine from its name inside curl_easy_duphandle. Reported-by: Anton Gerasimov Signed-of-by: Laurent Bonnans Fixes #2829 Closes #2833 - http2: make sure to send after RST_STREAM If this is the last stream on this connection, the RST_STREAM might not get pushed to the wire otherwise. Fixes #2882 Closes #2887 Researched-by: Michael Kaufmann - test1268: check the stderr output as "text" Follow-up to 099f37e9c57 Pointed-out-by: Marcel Raad - urldata: remove unused pipe_broke struct field This struct field is never set TRUE in any existing code path. This change removes the field completely. Closes #2871 - curl: warn the user if a given file name looks like an option ... simply because this is usually a sign of the user having omitted the file name and the next option is instead "eaten" by the parser as a file name. Add test1268 to verify Closes #2885 - http2: check nghttp2_session_set_stream_user_data return code Might help bug #2688 debugging Closes #2880 - travis: revert back to gcc-7 for coverage builds ... since the gcc-8 ones seem to fail frequently. Follow-up from b85207199544ca Closes #2886 - RELEASE-NOTES: synced ... and now listed in alphabetical order! - [Adrien brought this change] CMake: CMake config files are defining CURL_STATICLIB for static builds This change allows to use the CMake config files generated by Curl's CMake scripts for static builds of the library. The symbol CURL_STATIC lib must be defined to compile downstream, thus the config package is the perfect place to do so. Fixes #2817 Closes #2823 Reported-by: adnn on github Reviewed-by: Sergei Nikulov - TODO: host name sections in config files Kamil Dudka (14 Aug 2018) - ssh-libssh: fix infinite connect loop on invalid private key Added test 656 (based on test 604) to verify the fix. Bug: https://bugzilla.redhat.com/1595135 Closes #2879 - ssh-libssh: reduce excessive verbose output about pubkey auth The verbose message "Authentication using SSH public key file" was printed each time the ssh_userauth_publickey_auto() was called, which meant each time a packet was transferred over network because the API operates in non-blocking mode. This patch makes sure that the verbose message is printed just once (when the authentication state is entered by the SSH state machine). Daniel Stenberg (14 Aug 2018) - travis: disable h2 torture tests for "coverage" Since they started to fail almost 100% since a few days. Closes #2876 Marcel Raad (14 Aug 2018) - travis: update to GCC 8 Closes https://github.com/curl/curl/pull/2869 Daniel Stenberg (13 Aug 2018) - http: fix for tiny "HTTP/0.9" response Deal with tiny "HTTP/0.9" (header-less) responses by checking the status-line early, even before a full "HTTP/" is received to allow detecting 0.9 properly. Test 1266 and 1267 added to verify. Fixes #2420 Closes #2872 Kamil Dudka (13 Aug 2018) - docs: add disallow-username-in-url.d and haproxy-protocol.d on the list ... to make make the files appear in distribution tarballs Closes #2856 - .travis.yml: verify that man pages can be regenerated ... when curl is built from distribution tarball Closes #2856 Marcel Raad (11 Aug 2018) - Split non-portable part off test 1133 Split off testing file names with double quotes into new test 1158. Disable it for MSYS using a precheck as it doesn't support file names with double quotes (but Cygwin does, for example). Fixes https://github.com/curl/curl/issues/2796 Closes https://github.com/curl/curl/pull/2854 Jay Satiro (11 Aug 2018) - projects: Improve Windows perl detection in batch scripts - Determine if perl is in the user's PATH by running perl.exe. Prior to this change detection was done by checking the PATH for perl/ but that did not work in all cases (eg git install includes perl but not in perl/ path). Bug: https://github.com/curl/curl/pull/2865 Reported-by: Daniel Jeliński - [Michael Kaufmann brought this change] docs: Improve the manual pages of some callbacks - CURLOPT_HEADERFUNCTION: add newlines - CURLOPT_INTERLEAVEFUNCTION: fix the description of 'userdata' - CURLOPT_READDATA: mention crashes, same as in CURLOPT_WRITEDATA - CURLOPT_READFUNCTION: rename 'instream' to 'userdata' and explain how to set it Closes https://github.com/curl/curl/pull/2868 Marcel Raad (11 Aug 2018) - GCC: silence -Wcast-function-type uniformly Pointed-out-by: Rikard Falkeborn Closes https://github.com/curl/curl/pull/2860 - Silence GCC 8 cast-function-type warnings On Windows, casting between unrelated function types is fine and sometimes even necessary, so just use an intermediate cast to (void (*) (void)) to silence the warning as described in [0]. [0] https://gcc.gnu.org/onlinedocs/gcc-8.1.0/gcc/Warning-Options.html Closes https://github.com/curl/curl/pull/2860 Daniel Stenberg (11 Aug 2018) - CURLINFO_SIZE_UPLOAD: fix missing counter update Adds test 1522 for verification. Reported-by: cjmsoregan Fixes #2847 Closes #2864 - [Daniel Jelinski brought this change] Documentation: fix CURLOPT_SSH_COMPRESSION copy/paste bug Closes #2867 - RELEASE-NOTES: synced - openssl: fix potential NULL pointer deref in is_pkcs11_uri Follow-up to 298d2565e Coverity CID 1438387 Marcel Raad (10 Aug 2018) - travis: execute "set -eo pipefail" for coverage build Follow-up to 2de63ab179eb78630ee039ad94fb2a5423df522d and 0b87c963252d3504552ee0c8cf4402bd65a80af5. Closes https://github.com/curl/curl/pull/2862 Daniel Stenberg (10 Aug 2018) - lib1502: fix memory leak in torture test Reported-by: Marcel Raad Fixes #2861 Closes #2863 - docs: mention NULL is fine input to several functions Fixes #2837 Closes #2858 Reported-by: Markus Elfring - [Bas van Schaik brought this change] README.md: add LGTM.com code quality grade for C/C++ Closes #2857 - [Rikard Falkeborn brought this change] test1531: Add timeout Previously, the macro TEST_HANG_TIMEOUT was unused, but since there is looping going on, we might as well add timing instead of removing it. Closes #2853 - [Rikard Falkeborn brought this change] test1540: Remove unused macro TEST_HANG_TIMEOUT The macro has never been used, and it there is not really any place where it would make sense to add timing checks. Closes #2852 - [Rikard Falkeborn brought this change] asyn-thread: Remove unused macro The macro seems to never have been used. Closes #2852 - [Rikard Falkeborn brought this change] http_proxy: Remove unused macro SELECT_TIMEOUT Usage was removed in 5113ad0424044458ac497fa1458ebe0101356b22. Closes #2852 - [Rikard Falkeborn brought this change] formdata: Remove unused macro HTTPPOST_CONTENTTYPE_DEFAULT Its usage was removed in 84ad1fd3047815f9c6e78728bb351b828eac10b1. Closes #2852 - [Rikard Falkeborn brought this change] telnet: Remove unused macros TELOPTS and TELCMDS Their usage was removed in 3a145180cc754a5959ca971ef3cd243c5c83fc51. Closes #2852 - [Daniel Jelinski brought this change] openssl: fix debug messages Fixes #2806 Closes #2843 - configure: fix for -lpthread detection with OpenSSL and pkg-config ... by making sure it uses the -I provided by pkg-config! Reported-by: pszemus on github Fixes #2848 Closes #2850 - RELEASE-NOTES: synced - windows: follow up to the buffer-tuning 1ba1dba7 Somehow I didn't include the amended version of the previous fix. This is the missing piece. Pointed-out-by: Viktor Szakats - [Daniel Jelinski brought this change] windows: implement send buffer tuning Significantly enhances upload performance on modern Windows versions. Bug: https://curl.haxx.se/mail/lib-2018-07/0080.html Closes #2762 Fixes #2224 - [Anderson Toshiyuki Sasaki brought this change] ssl: set engine implicitly when a PKCS#11 URI is provided This allows the use of PKCS#11 URI for certificates and keys without setting the corresponding type as "ENG" and the engine as "pkcs11" explicitly. If a PKCS#11 URI is provided for certificate, key, proxy_certificate or proxy_key, the corresponding type is set as "ENG" if not provided and the engine is set to "pkcs11" if not provided. Acked-by: Nikos Mavrogiannopoulos Closes #2333 - [Ruslan Baratov brought this change] CMake: Respect BUILD_SHARED_LIBS Use standard CMake variable BUILD_SHARED_LIBS instead of introducing custom option CURL_STATICLIB. Use '-DBUILD_SHARED_LIBS=%SHARED%' in appveyor.yml. Reviewed-by: Sergei Nikulov Closes #2755 - [John Butterfield brought this change] cmake: bumped minimum version to 3.4 Closes #2753 - [John Butterfield brought this change] cmake: link curl to the OpenSSL targets instead of lib absolute paths Reviewed-by: Jakub Zakrzewski Reviewed-by: Sergei Nikulov Closes #2753 - travis: build darwinssl on macos 10.12 ... as building on 10.13.x before 10.13.4 leads to link errors. Assisted-by: Nick Zitzmann Fixes #2835 Closes #2845 - DEPRECATE: remove release date from 7.62.0 Since it will slip and the version is the important part there, not the date. - lib/Makefile: only do symbol hiding if told to This restores the ability to build a static lib with --disable-symbol-hiding to keep non-curl_ symbols. Researched-by: Dan Fandrich Reported-by: Ran Mozes Fixes #2830 Closes #2831 Marcel Raad (2 Aug 2018) - hostip: fix unused variable warning addresses is only used in an infof call, which is a macro expanding to nothing if CURL_DISABLE_VERBOSE_STRINGS is set. Daniel Stenberg (2 Aug 2018) - test1307: disabled Turns out that since we're using the native fnmatch function now when available, and they simply disagree on a huge number of test patterns that make it hard to test this function like this... Fixes #2825 - smb: don't mark it done in smb_do Follow-up to 09e401e01bf9. The SMB protocol handler needs to use its doing function too, which requires smb_do() to not mark itself as done... Closes #2822 - [Rikard Falkeborn brought this change] general: fix printf specifiers Closes #2818 - RELEASE-NOTES: synced - mailmap: Daniel Jelinski - [Harry Sintonen brought this change] HTTP: Don't attempt to needlessly decompress redirect body This change fixes a regression where redirect body would needlessly be decompressed even though it was to be ignored anyway. As it happens this causes secondary issues since there appears to be a bug in apache2 that it in certain conditions generates a corrupt zlib response. The regression was created by commit: dbcced8e32b50c068ac297106f0502ee200a1ebd Discovered-by: Harry Sintonen Closes #2798 - curl: use Content-Disposition before the "URL end" for -OJ Regression introduced in 7.61.0 Reported-by: Thomas Klausner Fixes #2783 Closes #2813 - [Daniel Jelinski brought this change] retry: return error if rewind was necessary but didn't happen Fixes #2801 Closes #2812 - http2: clear the drain counter in Curl_http2_done Reported-by: Andrei Virtosu Fixes #2800 Closes #2809 - smb: fix memory leak on early failure ... by making sure connection related data (->share) is stored in the connection and not in the easy handle. Detected by OSS-fuzz Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=9369 Fixes #2769 Closes #2810 - travis: run a 'make checksrc' too ... to make sure the examples are all checked. Closes #2811 Jay Satiro (29 Jul 2018) - examples/ephiperfifo: checksrc compliance - [Michael Kaufmann brought this change] sws: handle EINTR when calling select() Closes https://github.com/curl/curl/pull/2808 Daniel Stenberg (29 Jul 2018) - test1157: follow-up to 35ecffb9 Ignore the user-agent line. Pointed-out-by: Marcel Raad Michael Kaufmann (29 Jul 2018) - tests/http_pipe.py: Use /usr/bin/env to find python Daniel Stenberg (28 Jul 2018) - TODO: Support Authority Information Access certificate extension (AIA) Closes #2793 - conn_free: updated comment to clarify Let's call it disassociate instead of disconnect since the latter term is used so much for (TCP) connections already. - test1157: test -H from empty file Verifies bugfix #2797 - [Tobias Blomberg brought this change] curl: Fix segfault when -H @headerfile is empty The curl binary would crash if the -H command line option was given a filename to read using the @filename syntax but that file was empty. Closes #2797 - mime: check Curl_rand_hex's return code Bug: https://curl.haxx.se/mail/archive-2018-07/0015.html Reported-by: Jeffrey Walton Closes #2795 - [Josh Bialkowski brought this change] docs/examples: add hiperfifo example using linux epoll/timerfd Closes #2804 - [Darío Hereñú brought this change] docs/INSTALL.md: minor formatting fixes Closes #2794 - [Christopher Head brought this change] docs/CURLOPT_URL: fix indentation The statement, “The application does not have to keep the string around after setting this option,” appears to be indented under the RTMP paragraph. It actually applies to all protocols, not just RTMP. Eliminate the extra indentation. Closes #2788 - [Christopher Head brought this change] docs/CURLOPT_WRITEFUNCTION: size is always 1 For compatibility with `fwrite`, the `CURLOPT_WRITEFUNCTION` callback is passed two `size_t` parameters which, when multiplied, designate the number of bytes of data passed in. In practice, CURL always sets the first parameter (`size`) to 1. This practice is also enshrined in documentation and cannot be changed in future. The documentation states that the default callback is `fwrite`, which means `fwrite` must be a suitable function for this purpose. However, the documentation also states that the callback must return the number of *bytes* it successfully handled, whereas ISO C `fwrite` returns the number of items (each of size `size`) which it wrote. The only way these numbers can be equal is if `size` is 1. Since `size` is 1 and can never be changed in future anyway, document that fact explicitly and let users rely on it. Closes #2787 - [Carie Pointer brought this change] wolfSSL/CyaSSL: Fix memory leak in Curl_cyassl_random RNG structure must be freed by call to FreeRng after its use in Curl_cyassl_random. This call fixes Valgrind failures when running the test suite with wolfSSL. Closes #2784 - [Even Rouault brought this change] reuse_conn(): free old_conn->options This fixes a memory leak when CURLOPT_LOGIN_OPTIONS is used, together with connection reuse. I found this with oss-fuzz on GDAL and curl master: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=9582 I couldn't reproduce with the oss-fuzz original test case, but looking at curl source code pointed to this well reproducable leak. Closes #2790 Marcel Raad (25 Jul 2018) - [Daniel Jelinski brought this change] system_win32: fix version checking In the current version, VERSION_GREATER_THAN_EQUAL 6.3 will return false when run on windows 10.0. This patch addresses that error. Closes https://github.com/curl/curl/pull/2792 Daniel Stenberg (24 Jul 2018) - [Johannes Schindelin brought this change] auth: pick Bearer authentication whenever a token is available So far, the code tries to pick an authentication method only if user/password credentials are available, which is not the case for Bearer authentictation... Signed-off-by: Johannes Schindelin Closes #2754 - [Johannes Schindelin brought this change] auth: only ever pick CURLAUTH_BEARER if we *have* a Bearer token The Bearer authentication was added to cURL 7.61.0, but there is a problem: if CURLAUTH_ANY is selected, and the server supports multiple authentication methods including the Bearer method, we strongly prefer that latter method (only CURLAUTH_NEGOTIATE beats it), and if the Bearer authentication fails, we will never even try to attempt any other method. This is particularly unfortunate when we already know that we do not have any Bearer token to work with. Such a scenario happens e.g. when using Git to push to Visual Studio Team Services (which supports Basic and Bearer authentication among other methods) and specifying the Personal Access Token directly in the URL (this aproach is frequently taken by automated builds). Let's make sure that we have a Bearer token to work with before we select the Bearer authentication among the available authentication methods. Signed-off-by: Johannes Schindelin Closes #2754 Marcel Raad (22 Jul 2018) - test320: treat curl320.out file as binary Otherwise, LF line endings are converted to CRLF on Windows, but no conversion is done for the reply, so the test case fails. Closes https://github.com/curl/curl/pull/2776 Daniel Stenberg (22 Jul 2018) - vtls: set conn->data when closing TLS Follow-up to 1b76c38904f0. The VTLS backends that close down the TLS layer for a connection still needs a Curl_easy handle for the session_id cache etc. Fixes #2764 Closes #2771 Marcel Raad (21 Jul 2018) - tests: fixes for Windows line endlings Set mode="text" when line endings depend on the system representation. Closes https://github.com/curl/curl/pull/2772 - test214: disable MSYS2's POSIX path conversion for URL By default, the MSYS2 bash converts all backslashes to forward slashes in URLs. Disable this with MSYS2_ARG_CONV_EXCL for the test to pass. Ref https://github.com/msys2/msys2/wiki/Porting#filesystem-namespaces Daniel Stenberg (20 Jul 2018) - http2: several cleanups - separate easy handle from connections better - added asserts on a number of places - added sanity check of pipelines for debug builds Closes #2751 - smb_getsock: always wait for write socket too ... the protocol is doing read/write a lot, so it needs to write often even when downloading. A more proper fix could check for eactly when it wants to write and only ask for it then. Without this fix, an SMB download could easily get stuck when the event-driven API was used. Closes #2768 Marcel Raad (20 Jul 2018) - test1143: disable MSYS2's POSIX path conversion By default, the MSYS2 bash interprets http:/%HOSTIP:%HTTPPORT/want/1143 as a POSIX file list and converts it to a Windows file list. Disable this with MSYS2_ARG_CONV_EXCL for the test to pass. Ref https://github.com/msys2/msys2/wiki/Porting#filesystem-namespaces Closes https://github.com/curl/curl/pull/2765 Daniel Stenberg (18 Jul 2018) - RELEASE-NOTES: sync ... and work toward 7.61.1 - [Ruslan Baratov brought this change] CMake: Update scripts to use consistent style Closes #2727 Reviewed-by: Sergei Nikulov - header output: switch off all styles, not just unbold ... the "unbold" sequence doesn't work on the mac Terminal. Reported-by: Zero King Fixes #2736 Closes #2738 Nick Zitzmann (14 Jul 2018) - [Rodger Combs brought this change] darwinssl: add support for ALPN negotiation Marcel Raad (14 Jul 2018) - test1422: add required file feature curl configured with --enable-debug --disable-file currently complains on test1422: Info: Protocol "file" not supported or disabled in libcurl Make test1422 dependend on enabled FILE protocol to fix this. Fixes https://github.com/curl/curl/issues/2741 Closes https://github.com/curl/curl/pull/2742 Patrick Monnerat (12 Jul 2018) - content_encoding: accept up to 4 unknown trailer bytes after raw deflate data Some servers issue raw deflate data that may be followed by an undocumented trailer. This commit makes curl tolerate such a trailer of up to 4 bytes before considering the data is in error. Reported-by: clbr on github Fixes #2719 Daniel Stenberg (12 Jul 2018) - smb: fix memory-leak in URL parse error path Detected by OSS-Fuzz Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=9369 Closes #2740 Marcel Raad (12 Jul 2018) - schannel: enable CALG_TLS1PRF for w32api >= 5.1 The definition of CALG_TLS1PRF has been fixed in the 5.1 branch: https://osdn.net/projects/mingw/scm/git/mingw-org-wsl/commits/73aedcc0f2e6ba370de0d86ab878ad76a0dda7b5 Daniel Stenberg (12 Jul 2018) - docs/SECURITY-PROCESS: mention bounty, drop pre-notify + The hackerone bounty and its process - We don't and can't handle pre-notification - multi: always do the COMPLETED procedure/state It was previously erroneously skipped in some situations. libtest/libntlmconnect.c wrongly depended on wrong behavior (that it would get a zero timeout) when no handles are "running" in a multi handle. That behavior is no longer present with this fix. Now libcurl will always return a -1 timeout when all handles are completed. Closes #2733 - Curl_getoff_all_pipelines: improved for multiplexed On multiplexed connections, transfers can be removed from anywhere not just at the head as for pipelines. - ares: check for NULL in completed-callback - conn: remove the boolean 'inuse' field ... as the usage needs to be counted. - [Paul Howarth brought this change] openssl: assume engine support in 1.0.0 or later Commit 38203f1585da changed engine detection to be version-based, with a baseline of openssl 1.0.1. This does in fact break builds with openssl 1.0.0, which has engine support - the configure script detects that ENGINE_cleanup() is available - but doesn't get included to declare it. According to upstream documentation, engine support was added to mainstream openssl builds as of version 0.9.7: https://github.com/openssl/openssl/blob/master/README.ENGINE This commit drops the version test down to 1.0.0 as version 1.0.0d is the oldest version I have to test with. Closes #2732 Marcel Raad (11 Jul 2018) - schannel: fix MinGW compile break Original MinGW's w32api has a sytax error in its definition of CALG_TLS1PRF [0]. Don't use original MinGW w32api's CALG_TLS1PRF until this bug [1] is fixed. [0] https://osdn.net/projects/mingw/scm/git/mingw-org-wsl/blobs/d1d4a17e51a2b78e252ef0147d483267d56c90cc/w32api/include/wincrypt.h [1] https://osdn.net/projects/mingw/ticket/38391 Fixes https://github.com/curl/curl/pull/2721#issuecomment-403636043 Closes https://github.com/curl/curl/pull/2728 Daniel Stenberg (11 Jul 2018) - examples/crawler.c: move #ifdef to column 0 Apparently the C => HTML converter on the web site doesn't quite like it otherwise. Reported-by: Jeroen Ooms Version 7.61.0 (11 Jul 2018) Daniel Stenberg (11 Jul 2018) - release: 7.61.0 - TODO: Configurable loading of OpenSSL configuration file Closes #2724 - post303.d: clarify that this is an RFC violation ... and not the other way around, which this previously said. Reported-by: Vasiliy Faronov Fixes #2723 Closes #2726 - [Ruslan Baratov brought this change] CMake: remove redundant and old end-of-block syntax Reviewed-by: Jakub Zakrzewski Closes #2715 Jay Satiro (9 Jul 2018) - lib/curl_setup.h: remove unicode character Follow-up to 82ce416. Ref: https://github.com/curl/curl/commit/8272ec5#commitcomment-29646818 Daniel Stenberg (9 Jul 2018) - lib/curl_setup.h: remove unicode bom from 8272ec50f02 Marcel Raad (9 Jul 2018) - schannel: fix -Wsign-compare warning MinGW warns: /lib/vtls/schannel.c:219:64: warning: signed and unsigned type in conditional expression [-Wsign-compare] Fix this by casting the ptrdiff_t to size_t as we know it's positive. Closes https://github.com/curl/curl/pull/2721 - schannel: workaround for wrong function signature in w32api Original MinGW's w32api has CryptHashData's second parameter as BYTE * instead of const BYTE *. Closes https://github.com/curl/curl/pull/2721 - schannel: make more cipher options conditional They are not defined in the original MinGW's . Closes https://github.com/curl/curl/pull/2721 - curl_setup: include before Otherwise, only part of it gets pulled in through on original MinGW. Fixes https://github.com/curl/curl/issues/2361 Closes https://github.com/curl/curl/pull/2721 - examples: fix -Wformat warnings When size_t is not a typedef for unsigned long (as usually the case on Windows), GCC emits -Wformat warnings when using lu and lx format specifiers with size_t. Silence them with explicit casts to unsigned long. Closes https://github.com/curl/curl/pull/2721 Daniel Stenberg (9 Jul 2018) - smtp: use the upload buffer size for scratch buffer malloc ... not the read buffer size, as that can be set smaller and thus cause a buffer overflow! CVE-2018-0500 Reported-by: Peter Wu Bug: https://curl.haxx.se/docs/adv_2018-70a2.html - [Dave Reisner brought this change] scripts: include _curl as part of CLEANFILES Closes #2718 - [Nick Zitzmann brought this change] darwinssl: allow High Sierra users to build the code using GCC ...but GCC users lose out on TLS 1.3 support, since we can't weak-link enumeration constants. Fixes #2656 Closes #2703 - [Ruslan Baratov brought this change] CMake: Remove unused 'output_var' from 'collect_true' Variable 'output_var' is not used and can be removed. Function 'collect_true' renamed to 'count_true'. - [Ruslan Baratov brought this change] CMake: Remove unused functions Closes #2711 - KNOWN_BUGS: Stick to same family over SOCKS proxy - libssh: goto DISCONNECT state on error, not SSH_SESSION_FREE ... because otherwise not everything get closed down correctly. Fixes #2708 Closes #2712 - libssh: include line number in state change debug messages Closes #2713 - KNOWN_BUGS: Borland support is dropped, AIX problem is too old - [Jeroen Ooms brought this change] example/crawler.c: simple crawler based on libxml2 Closes #2706 - RELEASE-NOTES: synced - DEPRECATE: include year when specifying date - DEPRECATE: linkified - DEPRECATE: mention the PR that disabled axTLS - docs/DEPRECATE.md: spelling and minor formatting - DEPRECATE: new doc describing planned item removals Closes #2704 - [Gisle Vanem brought this change] telnet: fix clang warnings telnet.c(1401,28): warning: cast from function call of type 'int' to non-matching type 'HANDLE' (aka 'void *') [-Wbad-function-cast] Fixes #2696 Closes #2700 - docs: fix missed option name markups - [Gaurav Malhotra brought this change] openssl: Remove some dead code Closes #2698 - openssl: make the requested TLS version the *minimum* wanted The code treated the set version as the *exact* version to require in the TLS handshake, which is not what other TLS backends do and probably not what most people expect either. Reported-by: Andreas Olsson Assisted-by: Gaurav Malhotra Fixes #2691 Closes #2694 - RELEASE-NOTES: synced - openssl: allow TLS 1.3 by default Reported-by: Andreas Olsson Fixes #2692 Closes #2693 - [Adrian Peniak brought this change] CURLINFO_TLS_SSL_PTR.3: improve the example The previous example was a little bit confusing, because SSL* structure (or other "in use" SSL connection pointer) is not accessible after the transfer is completed, therefore working with the raw TLS library specific pointer needs to be done during transfer. Closes #2690 - travis: add a build using the synchronous name resolver ... since default uses the threaded one and we test the c-ares build already. Closes #2689 - configure: remove CURL_CHECK_NI_WITHSCOPEID too Since it isn't used either and requires the getnameinfo check Follow-up to 0aeca41702d2 - getnameinfo: not used Closes #2687 - easy_perform: use *multi_timeout() to get wait times ... and trim the threaded Curl_resolver_getsock() to return zero millisecond wait times during the first three milliseconds so that localhost or names in the OS resolver cache gets detected and used faster. Closes #2685 Max Dymond (27 Jun 2018) - configure: Add dependent libraries after crypto The linker is pretty dumb and processes things left to right, keeping a tally of symbols it hasn't resolved yet. So, we need -ldl to appear after -lcrypto otherwise the linker won't find the dl functions. Closes #2684 Daniel Stenberg (27 Jun 2018) - GOVERNANCE: linkify, changed some titles - GOVERNANCE: add maintainer details/duties - url: check Curl_conncache_add_conn return code ... it was previously unchecked in two places and thus errors could remain undetected and cause trouble. Closes #2681 - include/README: remove "hacking" advice, not the right place - RELEASE-NOTES: synced - CURLOPT_SSL_VERIFYPEER.3: fix syntax mistake Follow-up to b6a16afa0aa5 - netrc: use a larger buffer ... to work with longer passwords etc. Grow it from a 256 to a 4096 bytes buffer. Reported-by: Dario Nieuwenhuis Fixes #2676 Closes #2680 - [Patrick Schlangen brought this change] CURLOPT_SSL_VERIFYPEER.3: Add performance note Closes #2673 - [Javier Blazquez brought this change] multi: fix crash due to dangling entry in connect-pending list Fixes #2677 Closes #2679 - ConnectionExists: make sure conn->data is set when "taking" a connection Follow-up to 2c15693. Bug #2674 Closes #2675 - [Kevin R. Bulgrien brought this change] system.h: fix for gcc on 32 bit OpenServer Bug: https://curl.haxx.se/mail/lib-2018-06/0100.html - [Raphael Gozzo brought this change] cmake: allow multiple SSL backends This will make possible to select the SSL backend (using curl_global_sslset()) even when the libcurl is built using CMake Closes #2665 - url: fix dangling conn->data pointer By masking sure to use the *current* easy handle with extracted connections from the cache, and make sure to NULLify the ->data pointer when the connection is put into the cache to make this mistake easier to detect in the future. Reported-by: Will Dietz Fixes #2669 Closes #2672 - CURLOPT_INTERFACE.3: interface names not supported on Windows - travis: run more tests for coverage check ... run a few more tortured based and run all tests event-based. Closes #2664 - multi: fix memory leak when stopped during name resolve When the application just started the transfer and then stops it while the name resolve in the background thread hasn't completed, we need to wait for the resolve to complete and then cleanup data accordingly. Enabled test 1553 again and added test 1590 to also check when the host name resolves successfully. Detected by OSS-fuzz. Closes #1968 Viktor Szakats (15 Jun 2018) - maketgz: delete .bak files, fix indentation Ref: https://github.com/curl/curl/pull/2660 Closes https://github.com/curl/curl/pull/2662 Daniel Stenberg (15 Jun 2018) - runtests.pl: remove debug leftover from bb9a340c73f3 - curl-confopts.m4: fix typo from ed224f23d5beb Fixes my local configure to detect a custom installed c-ares without pkgconfig. - docs/RELEASE-PROCEDURE.md: renamed to use .md extension Closes #2663 - RELEASE-PROCEDURE: gpg sign the tags - RELEASE-NOTES: synced - CURLOPT_HTTPAUTH.3: CURLAUTH_BEARER was added in 7.61.0 - [Mamta Upadhyay brought this change] maketgz: fix sed issues on OSX maketgz creates release tarballs and removes the -DEV string in curl version (e.g. 7.58.0-DEV), else -DEV shows up on command line when curl is run. maketgz works fine on linux but fails on OSX. Problem is with the sed commands that use option -i without an extension. Maketgz expects GNU sed instead of BSD and this simply won't work on OSX. Adding a backup extension .bak after -i fixes this issue Running the script as if on OSX gives this error: sed: -e: No such file or directory Adding a .bak extension resolves it Closes #2660 - configure: enhance ability to detect/build with static openssl Fix the -ldl and -ldl + -lpthread checks for OpenSSL, necessary for building with static libs without pkg-config. Reported-by: Marcel Raad Fixes #2199 Closes #2659 - configure: use pkg-config for c-ares detection First check if there's c-ares information given as pkg-config info and use that as first preference. Reported-by: pszemus on github Fixes #2203 Closes #2658 - GOVERNANCE.md: explains how this project is run Closes #2657 - KNOWN_BUGS: NTLM doen't support password with § character Closes #2120 - KNOWN_BUGS: slow connect to localhost on Windows Closes #2281 - [Matteo Bignotti brought this change] mk-ca-bundle.pl: make -u delete certdata.txt if found not changed certdata.txt should be deleted also when the process is interrupted by "same certificate downloaded, exiting" The certdata.txt is currently kept on disk even if you give the -u option Closes #2655 - progress: remove a set of unused defines Reported-by: Peter Wu Closes #2654 - TODO: "Option to refuse usernames in URLs" done Implemented by Björn in 946ce5b61f - [Lyman Epp brought this change] Curl_init_do: handle NULL connection pointer passed in Closes #2653 - runtests: support variables in ... and make use of that to make 1455 work better without using a fixed local port number. Fixes #2649 Closes #2650 - Curl_debug: remove dead printhost code The struct field is never set (since 5e0d9aea3) so remove the use of it and remove the connectdata pointer from the prototype. Reported-by: Tejas Bug: https://curl.haxx.se/mail/lib-2018-06/0054.html Closes #2647 Viktor Szakats (12 Jun 2018) - schannel: avoid incompatible pointer warning with clang-6.0: ``` vtls/schannel_verify.c: In function 'add_certs_to_store': vtls/schannel_verify.c:212:30: warning: passing argument 11 of 'CryptQueryObject' from incompatible pointer type [-Wincompatible-pointer-types] &cert_context)) { ^ In file included from /usr/share/mingw-w64/include/schannel.h:10:0, from /usr/share/mingw-w64/include/schnlsp.h:9, from vtls/schannel.h:29, from vtls/schannel_verify.c:40: /usr/share/mingw-w64/include/wincrypt.h:4437:26: note: expected 'const void **' but argument is of type 'CERT_CONTEXT ** {aka struct _CERT_CONTEXT **}' WINIMPM WINBOOL WINAPI CryptQueryObject (DWORD dwObjectType, const void *pvObject, DWORD dwExpectedContentTypeFlags, DWORD dwExpectedFormatTypeFlags, DWORD dwFlags, ^~~~~~~~~~~~~~~~ ``` Ref: https://msdn.microsoft.com/library/windows/desktop/aa380264 Closes https://github.com/curl/curl/pull/2648 Daniel Stenberg (12 Jun 2018) - [Robert Prag brought this change] schannel: support selecting ciphers Given the contstraints of SChannel, I'm exposing these as the algorithms themselves instead; while replicating the ciphersuite as specified by OpenSSL would have been preferable, I found no way in the SChannel API to do so. To use this from the commandline, you need to pass the names of contants defining the desired algorithms. For example, curl --ciphers "CALG_SHA1:CALG_RSA_SIGN:CALG_RSA_KEYX:CALG_AES_128:CALG_DH_EPHEM" https://github.com The specific names come from wincrypt.h Closes #2630 - [Bernhard M. Wiedemann brought this change] test 46: make test pass after 2025 shifting the expiry date to 2037 for now to be before the possibly problematic year 2038 similar in spirit to commit e6293cf8764e9eecb Closes #2646 - [Marian Klymov brought this change] cppcheck: fix warnings - Get rid of variable that was generating false positive warning (unitialized) - Fix issues in tests - Reduce scope of several variables all over etc Closes #2631 - openssl: assume engine support in 1.0.1 or later Previously it was checked for in configure/cmake, but that would then leave other build systems built without engine support. While engine support probably existed prior to 1.0.1, I decided to play safe. If someone experience a problem with this, we can widen the version check. Fixes #2641 Closes #2644 - RELEASE-NOTES: synced - RELEASE-PROCEDURE: update the release calendar for 2019 - [Gisle Vanem brought this change] boringssl + schannel: undef X509_NAME in lib/schannel.h Fixes the build problem when both boringssl and schannel are enabled. Fixes #2634 Closes #2643 - [Vladimir Kotal brought this change] mk-ca-bundle.pl: leave certificate name untouched in decode() Closes #2640 - [Rikard Falkeborn brought this change] tests/libtests/Makefile.am: Add lib1521.c to CLEANFILES This removes the generated lib1521.c when running make clean. Closes #2633 - [Rikard Falkeborn brought this change] tests/libtest: Add lib1521 to nodist_SOURCES Since 467da3af0, lib1521.c is generated instead of checked in. According to the commit message, the intention was to remove it from the tarball as well. However, it is still present when running make dist. To remove it, add it to nodist_lib1521_SOURCES. This also means there is no need for the manually added dist-rule in the Makefile. Also update CMakelists.txt to handle the fact that we now may have nodist_SOURCES. - [Stephan Mühlstrasser brought this change] system.h: add support for IBM xlc C compiler Added a section to system.h guarded with __xlc__ for the IBM xml C compiler. Before this change the section titled 'generic "safe guess" on old 32 bit style' was used, which resulted in a wrong definition of CURL_TYPEOF_CURL_SOCKLEN_T, and for 64-bit also CURL_TYPEOF_CURL_OFF_T was wrong. Compilation warnings fixed with this change: CC libcurl_la-ftp.lo "ftp.c", line 290.55: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed. "ftp.c", line 293.48: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed. "ftp.c", line 1070.49: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed. "ftp.c", line 1154.53: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed. "ftp.c", line 1187.51: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed. CC libcurl_la-connect.lo "connect.c", line 448.56: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed. "connect.c", line 516.66: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed. "connect.c", line 687.55: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed. "connect.c", line 696.55: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed. CC libcurl_la-tftp.lo "tftp.c", line 1115.33: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed. Closes #2637 - cmdline-opts/cert-type.d: mention "p12" as a recognized type as well Viktor Szakats (3 Jun 2018) - spelling fixes Detected using the `codespell` tool (version 1.13.0). Also secure and fix an URL. Daniel Stenberg (2 Jun 2018) - axtls: follow-up spell fix of comment - axTLS: not considered fit for use URL: https://curl.haxx.se/mail/lib-2018-06/0000.html This is step one. It adds #error statements that require source edits to make curl build again if asked to use axTLS. At a later stage we might remove the axTLS specific code completely. Closes #2628 - build: remove the Borland specific makefiles According to the user survey 2018, not even one out of 670 users use them. Nobody on the mailing list spoke up for them either. Closes #2629 - curl_addrinfo: use same #ifdef conditions in source as header ... for curl_dofreeaddrinfo - multi: remove a DEBUGF() ... it might call infof() with a NULL first argument that isn't harmful but makes it not do anything. The infof() line is not very useful anymore, it has served it purpose. Good riddance! Fixes #2627 - [Alibek.Jorajev brought this change] CURLOPT_RESOLVE: always purge old entry first If there's an existing entry using the selected name. Closes #2622 - fnmatch: use the system one if available If configure detects fnmatch to be available, use that instead of our custom one for FTP wildcard pattern matching. For standard compliance, to reduce our footprint and to use already well tested and well exercised code. A POSIX fnmatch behaves slightly different than the internal function for a few test patterns currently and the macOS one yet slightly different. Test case 1307 is adjusted for these differences. Closes #2626 Patrick Monnerat (31 May 2018) - os400: add new option in ILE/RPG binding Follow-up to commit 946ce5b Daniel Stenberg (31 May 2018) - tests/libtest/.gitignore: follow-up fix to ignore lib5* too - KNOWN_BUGS: CURL_GLOBAL_SSL Closes #2276 - [Bernhard Walle brought this change] configure: check for declaration of getpwuid_r On our x86 Android toolchain, getpwuid_r is implemented but the header is missing: netrc.c:81:7: error: implicit declaration of function 'getpwuid_r' [-Werror=implicit-function-declaration] Unfortunately, the function is used in curl_ntlm_wb.c, too, so I moved the prototype to curl_setup.h. Signed-off-by: Bernhard Walle Closes #2609 - [Rikard Falkeborn brought this change] tests: update .gitignore for libtests Closes #2624 - [Rikard Falkeborn brought this change] strictness: correct {infof, failf} format specifiers Closes #2623 - [Björn Stenberg brought this change] option: disallow username in URL Adds CURLOPT_DISALLOW_USERNAME_IN_URL and --disallow-username-in-url. Makes libcurl reject URLs with a username in them. Closes #2340 - libcurl-security.3: improved layout for two rememdy lists - libcurl-security.3: refer to URL instead of in-source markdown file Viktor Szakats (30 May 2018) - curl.rc: embed manifest for correct Windows version detection * enable it in `src/Makefile.m32` * enable it in `winbuild/MakefileBuild.vc` if a custom manifest is _not_ enabled via the existing `EMBED_MANIFEST` option * enable it for all Windows CMake builds (also disable the built-in minimal manifest, added by CMake by default.) For other build systems, add the `-DCURL_EMBED_MANIFEST` option to the list of RC (Resource Compiler) flags to enable the manifest included in `src/curl.rc`. This may require to disable whatever automatic or other means in which way another manifest is added to `curl.exe`. Notice that Borland C doesn't support this method due to a long-pending resource compiler bug. Watcom C may also not handle it correctly when the `-zm` `wrc` option is used (this option may be unnecessary though) and regardless of options in certain earlier revisions of the 2.0 beta version. Closes https://github.com/curl/curl/pull/1221 Fixes https://github.com/curl/curl/issues/2591 Patrick Monnerat (30 May 2018) - os400: sync EBCDIC wrappers and ILE/RPG binding with latest options - os400: implement mime api EBCDIC wrappers Also sync ILE/RPG binding to define the new functions. Daniel Stenberg (29 May 2018) - setopt: add TLS 1.3 ciphersuites Adds CURLOPT_TLS13_CIPHERS and CURLOPT_PROXY_TLS13_CIPHERS. curl: added --tls13-ciphers and --proxy-tls13-ciphers Fixes #2435 Reported-by: zzq1015 on github Closes #2607 - configure: override AR_FLAGS to silence warning The automake default ar flags are 'cru', but the 'u' flag in there causes warnings on many modern Linux distros. Removing 'u' may have a minor performance impact on older distros but should not cause harm. Explained on the automake mailing list already back in April 2015: https://www.mail-archive.com/automake-patches@gnu.org/msg07705.html Reported-by: elephoenix on github Fixes #2617 Closes #2619 Sergei Nikulov (29 May 2018) - cmake: fixed comments in compile checks code Daniel Stenberg (29 May 2018) - INSTALL: LDFLAGS=-Wl,-R/usr/local/ssl/lib ... the older description doesn't work Reported-by: Peter Varga Fixes #2615 Closes #2616 - [Will Dietz brought this change] KNOWN_BUGS: restore text regarding #2101. This was added earlier but appears to have been removed accidentally. AFAICT this is very much still an issue. ----- I say "accidentally" because the text seems to have harmlessly snuck into [1] (which makes no mention of it). [1] was later reverted for unspecified reasons in [2], presumably because the mentioned issue was fixed or invalid. [1] de9fac00c40db321d44fa6fbab6eb62ec4c83998 [2] 16d1f369403cbb04bd7b085eabbeebf159473fc2 Closes #2618 - fnmatch: insist on escaped bracket to match A non-escaped bracket ([) is for a character group - as documented. It will *not* match an individual bracket anymore. Test case 1307 updated accordingly to match. Problem detected by OSS-Fuzz, although this fix is probably not a final fix for the notorious timeout issues. Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8525 Closes #2614 Patrick Monnerat (28 May 2018) - psl: use latest psl and refresh it periodically The latest psl is cached in the multi or share handle. It is refreshed before use after 72 hours. New share lock CURL_LOCK_DATA_PSL controls the psl cache sharing. If the latest psl is not available, the builtin psl is used. Reported-by: Yaakov Selkowitz Fixes #2553 Closes #2601 Daniel Stenberg (28 May 2018) - [Fabrice Fontaine brought this change] configure: fix ssh2 linking when built with a static mbedtls The ssh2 pkg-config file could contain the following lines when build with a static version of mbedtls: Libs: -L${libdir} -lssh2 /xxx/libmbedcrypto.a Libs.private: /xxx/libmbedcrypto.a This static mbedtls library must be used to correctly detect ssh2 support and this library must be copied in libcurl.pc otherwise compilation of any application (such as upmpdcli) with libcurl will fail when trying to found mbedtls functions included in libssh2. So, replace pkg-config --libs-only-l by pkg-config --libs. Fixes: - http://autobuild.buildroot.net/results/43e24b22a77f616d6198c10435dcc23cc3b9088a Signed-off-by: Fabrice Fontaine Closes #2613 - RELEASE-NOTES: synced - [Bernhard Walle brought this change] cmake: check for getpwuid_r The autotools-based build system does it, so we do it also in CMake. Bug: #2609 Signed-off-by: Bernhard Walle - cmdline-opts/gen.pl: warn if mutexes: or see-also: list non-existing options - [Frank Gevaerts brought this change] curl.1: Fix cmdline-opts reference errors. --data, --form, and --ntlm were declared to be mutually exclusive with non-existing options. --data and --form referred to --upload (which is short for --upload-file and therefore did work, so this one was merely a bit confusing), --ntlm referred to --negotiated instead of --negotiate. Closes #2612 - [Frank Gevaerts brought this change] docs: fix cmdline-opts metadata headers case consistency. Almost all headers start with an uppercase letter, but some didn't. - mailmap: Max Savenkov Sergei Nikulov (28 May 2018) - [Max Savenkov brought this change] Fix the test for fsetxattr and strerror_r tests in CMake to work without compiling Daniel Stenberg (27 May 2018) - mailmap: a Richard Alcock fixup - [Richard Alcock brought this change] schannel: add failf calls for client certificate failures Closes #2604 - [Richard Alcock brought this change] winbuild: In MakefileBuild.vc fix typo DISTDIR->DIRDIST Change requirement from $(DISTDIR) to $(DIRDIST) closes #2603 - [Richard Alcock brought this change] winbuild: only delete OUTFILE if it exists This removes the slightly annoying "Could not file LIBCURL_OBJS.inc" and "Could not find CURL_OBJS.inc.inc" message when building into a clean folder. closes #2602 - [Alejandro R. Sedeño brought this change] content_encoding: handle zlib versions too old for Z_BLOCK Fallback on Z_SYNC_FLUSH when Z_BLOCK is not available. Fixes #2606 Closes #2608 - multi: provide a socket to wait for in Curl_protocol_getsock ... even when there's no protocol specific handler setup. Bug: https://curl.haxx.se/mail/lib-2018-05/0062.html Reported-by: Sean Miller Closes #2600 - [Linus Lewandowski brought this change] httpauth: add support for Bearer tokens Closes #2102 - TODO: CURLINFO_PAUSE_STATE Closes #2588 Sergei Nikulov (24 May 2018) - cmake: set -d postfix for debug builds if not specified using -DCMAKE_DEBUG_POSTFIX explicitly fixes #2121, obsoletes #2384 Daniel Stenberg (23 May 2018) - configure: add basic test of --with-ssl prefix When given a prefix, the $PREFIX_OPENSSL/lib/openssl.pc or $PREFIX_OPENSSL/include/openssl/ssl.h files must be present or cause an error. Helps users detect when giving configure the wrong path. Reported-by: Oleg Pudeyev Assisted-by: Per Malmberg Fixes #2580 Patrick Monnerat (22 May 2018) - http resume: skip body if http code 416 (range error) is ignored. This avoids appending error data to already existing good data. Test 92 is updated to match this change. New test 1156 checks all combinations of --range/--resume, --fail, Content-Range header and http status code 200/416. Fixes #1163 Reported-By: Ithubg on github Closes #2578 Daniel Stenberg (22 May 2018) - tftp: make sure error is zero terminated before printfing it - configure: add missing m4/ax_compile_check_sizeof.m4 follow-up to mistake in 6876ccf90b4 Jay Satiro (22 May 2018) - [Johannes Schindelin brought this change] schannel: make CAinfo parsing resilient to CR/LF OpenSSL has supported --cacert for ages, always accepting LF-only line endings ("Unix line endings") as well as CR/LF line endings ("Windows line endings"). When we introduced support for --cacert also with Secure Channel (or in cURL speak: "WinSSL"), we did not take care to support CR/LF line endings, too, even if we are much more likely to receive input in that form when using Windows. Let's fix that. Happily, CryptQueryObject(), the function we use to parse the ca-bundle, accepts CR/LF input already, and the trailing LF before the END CERTIFICATE marker catches naturally any CR/LF line ending, too. So all we need to care about is the BEGIN CERTIFICATE marker. We do not actually need to verify here that the line ending is CR/LF. Just checking for a CR or an LF is really plenty enough. Signed-off-by: Johannes Schindelin Closes https://github.com/curl/curl/pull/2592 Daniel Stenberg (22 May 2018) - CURLOPT_ACCEPT_ENCODING.3: add brotli and clarify a bit - RELEASE-NOTES: synced - KNOWN_BUGS: mention the -O with %-encoded file names Closes #2573 - checksrc: make sure sizeof() is used *with* parentheses ... and unify the source code to adhere. Closes #2563 - curl: added --styled-output It is enabled by default, so --no-styled-output will switch off the detection/use of bold headers. Closes #2538 - curl: show headers in bold The feature is only enabled if the output is believed to be a tty. -J: There's some minor differences and improvements in -J handling, as now J should work with -i and it actually creates a file first using the initial name and then *renames* that to the one found in Content-Disposition (if any). -i: only shows headers for HTTP transfers now (as documented). Previously it would also show for pieces of the transfer that were HTTP (for example when doing FTP over a HTTP proxy). -i: now shows trailers as well. Previously they were not shown at all. --libcurl: the CURLOPT_HEADER is no longer set, as the header output is now done in the header callback. - configure: compile-time SIZEOF checks ... instead of exeucting code to get the size. Removes the use of LD_LIBRARY_PATH for this. Fixes #2586 Closes #2589 Reported-by: Bernhard Walle - configure: replace AC_TRY_RUN with CURL_RUN_IFELSE ... and export LD_LIBRARY_PATH properly. This is a follow-up from 2d4c215. Fixes #2586 Reported-by: Bernhard Walle - docs: clarify CURLOPT_HTTPGET somewhat Reported-by: bsammon on github Fixes #2590 - curl_fnmatch: only allow two asterisks for matching The previous limit of 5 can still end up in situation that takes a very long time and consumes a lot of CPU. If there is still a rare use case for this, a user can provide their own fnmatch callback for a version that allows a larger set of wildcards. This commit was triggered by yet another OSS-Fuzz timeout due to this. Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8369 Closes #2587 - checksrc: fix too long line follow-up to e05ad5d - [Aleks brought this change] docs: mention HAproxy protocol "version 1" ...as there's also a version 2. Closes #2579 - examples/progressfunc: make it build on older libcurls This example was changed in ce2140a8c1 to use the new microsecond based getinfo option. This change makes it conditionally keep using the older option so that the example still builds with older libcurl versions. Closes #2584 - stub_gssapi: fix numerous 'unused parameter' warnings follow-up to d9e92fd9fd1d - [Philip Prindeville brought this change] getinfo: add microsecond precise timers for various intervals Provide a set of new timers that return the time intervals using integer number of microseconds instead of floats. The new info names are as following: CURLINFO_APPCONNECT_TIME_T CURLINFO_CONNECT_TIME_T CURLINFO_NAMELOOKUP_TIME_T CURLINFO_PRETRANSFER_TIME_T CURLINFO_REDIRECT_TIME_T CURLINFO_STARTTRANSFER_TIME_T CURLINFO_TOTAL_TIME_T Closes #2495 - openssl: acknowledge --tls-max for default version too ... previously it only used the max setting if a TLS version was also explicitly asked for. Reported-by: byte_bucket Fixes #2571 Closes #2572 - bump: start working on the pending 7.61.0 - [Dagobert Michelsen brought this change] tests/libtest/Makefile: Do not unconditionally add gcc-specific flags The warning flag leads e.g. Sun Studio compiler to bail out. Closes #2576 - schannel_verify: fix build for non-schannel Jay Satiro (16 May 2018) - rand: fix typo - schannel: disable manual verify if APIs not available .. because original MinGW and old compilers do not have the Windows API definitions needed to support manual verification. - [Archangel_SDY brought this change] schannel: disable client cert option if APIs not available Original MinGW targets Windows 2000 by default, which lacks some APIs and definitions for this feature. Disable it if these APIs are not available. Closes https://github.com/curl/curl/pull/2522 Version 7.60.0 (15 May 2018) Daniel Stenberg (15 May 2018) - RELEASE-NOTES: 7.60.0 release - THANKS: added people from the curl 7.60.0 release - docs/libcurl/index.html: removed The HTML files are long gone from the dist, now remove the last HTML file pointing to those missing files. d - [steini2000 brought this change] http2: remove unused variable Closes #2570 - [steini2000 brought this change] http2: use easy handle of stream for logging - gcc: disable picky gcc-8 function pointer warnings in two places Reported-by: Rikard Falkeborn Bug: #2560 Closes #2569 - http2: use the correct function pointer typedef Fixes gcc-8 picky compiler warnings Reported-by: Rikard Falkeborn Bug: #2560 Closes #2568 - CODE_STYLE: mention return w/o parens, but sizeof with ... and remove the github markdown syntax so that it renders better on the web site. Also, don't use back-ticks inlined to allow the CSS to highlight source code better. - [Rikard Falkeborn brought this change] examples: Fix format specifiers Closes #2561 - [Rikard Falkeborn brought this change] tool: Fix format specifiers - [Rikard Falkeborn brought this change] ntlm: Fix format specifiers - [Rikard Falkeborn brought this change] tests: Fix format specifiers - [Rikard Falkeborn brought this change] lib: Fix format specifiers - contributors.sh: use "on github", not at - http2: getsock fix for uploads When there's an upload in progress, make sure to wait for the socket to become writable. Detected-by: steini2000 on github Bug: #2520 Closes #2567 - pingpong: fix response cache memcpy overflow Response data for a handle with a large buffer might be cached and then used with the "closure" handle when it has a smaller buffer and then the larger cache will be copied and overflow the new smaller heap based buffer. Reported-by: Dario Weisser CVE: CVE-2018-1000300 Bug: https://curl.haxx.se/docs/adv_2018-82c2.html - http: restore buffer pointer when bad response-line is parsed ... leaving the k->str could lead to buffer over-reads later on. CVE: CVE-2018-1000301 Assisted-by: Max Dymond Detected by OSS-Fuzz. Bug: https://curl.haxx.se/docs/adv_2018-b138.html Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=7105 Patrick Monnerat (13 May 2018) - cookies: do not take cookie name as a parameter RFC 6265 section 4.2.1 does not set restrictions on cookie names. This is a follow-up to commit 7f7fcd0. Also explicitly check proper syntax of cookie name/value pair. New test 1155 checks that cookie names are not reserved words. Reported-By: anshnd at github Fixes #2564 Closes #2566 Daniel Stenberg (12 May 2018) - smb: reject negative file sizes Assisted-by: Max Dymond Detected by OSS-Fuzz Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8245 - setup_transfer: deal with both sockets being -1 Detected by Coverity; CID 1435559. Follow-up to f8d608f38d00. It would index the array with -1 if neither index was a socket. - travis: add build using NSS Closes #2558 - [Sunny Purushe brought this change] openssl: change FILE ops to BIO ops To make builds with VS2015 work. Recent changes in VS2015 _IOB_ENTRIES handling is causing problems. This fix changes the OpenSSL backend code to use BIO functions instead of FILE I/O functions to circumvent those problems. Closes #2512 - travis: add a build using WolfSSL Assisted-by: Dan Fandrich Closes #2528 - RELEASE-NOTES: typo - RELEASE-NOTES: synced - [Daniel Gustafsson brought this change] URLs: fix one more http url This file wasn't included in commit 4af40b3646d3b09 which updated all haxx.se http urls to https. The file was committed prior to that update, but may have been merged after it and hence didn't get updated. Closes #2550 - github/lock: auto-lock closed issues after 90 days of inactivity - vtls: fix missing commas follow-up to e66cca046cef - vtls: use unified "supports" bitfield member in backends ... instead of previous separate struct fields, to make it easier to extend and change individual backends without having to modify them all. closes #2547 - transfer: don't unset writesockfd on setup of multiplexed conns Curl_setup_transfer() can be called to setup a new individual transfer over a multiplexed connection so it shouldn't unset writesockfd. Bug: #2520 Closes #2549 - [Frank Gevaerts brought this change] configure: put CURLDEBUG and DEBUGBUILD in lib/curl_config.h They are removed from the compiler flags. This ensures that make dependency tracking will force a rebuild whenever configure --enable-debug or --enable-curldebug changes. Closes #2548 - http: don't set the "rewind" flag when not uploading anything It triggers an assert. Detected by OSS-Fuzz Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8144 Closes #2546 - travis: add an mbedtls build Closes #2531 - configure: only check for CA bundle for file-using SSL backends When only building with SSL backends that don't use the CA bundle file (by default), skip the check. Fixes #2543 Fixes #2180 Closes #2545 - ssh-libssh.c: fix left shift compiler warning ssh-libssh.c:2429:21: warning: result of '1 << 31' requires 33 bits to represent, but 'int' only has 32 bits [-Wshift-overflow=] 'len' will never be that big anyway so I converted the run-time check to a regular assert. - [Stephan Mühlstrasser brought this change] URL: fix ASCII dependency in strcpy_url and strlen_url Commit 3c630f9b0af097663a64e5c875c580aa9808a92b partially reverted the changes from commit dd7521bcc1b7a6fcb53c31f9bd1192fcc884bd56 because of the problem that strcpy_url() was modified unilaterally without also modifying strlen_url(). As a consequence strcpy_url() was again depending on ASCII encoding. This change fixes strlen_url() and strcpy_url() in parallel to use a common host-encoding independent criterion for deciding whether an URL character must be %-escaped. Closes #2535 - [Denis Ollier brought this change] docs: remove extraneous commas in man pages Closes #2544 - RELEASE-NOTES: synced - Revert "TODO: remove configure --disable-pthreads" This reverts commit d5d683a97f9765bddfd964fe32e137aa6e703ed3. --disable-pthreads can be used to disable pthreads and get the threaded resolver to use the windows threading when building with mingw. - vtls: don't define MD5_DIGEST_LENGTH for wolfssl ... as it defines it (too) - TODO: remove configure --disable-pthreads Jay Satiro (2 May 2018) - [David Garske brought this change] wolfssl: Fix non-blocking connect Closes https://github.com/curl/curl/pull/2542 Daniel Stenberg (30 Apr 2018) - CURLOPT_URL.3: add ENCODING section [ci skip] Feedback-by: Michael Kilburn - KNOWN_BUGS: Client cert with Issuer DN differs between backends Closes #1411 - KNOWN_BUGS: Passive transfer tries only one IP address Closes #1508 - KNOWN_BUGS: --upload-file . hang if delay in STDIN Closes #2051 - KNOWN_BUGS: Connection information when using TCP Fast Open Closes #1332 - travis: enable libssh2 on both macos and Linux It seems to not be detected by default anymore (which is a bug I believe) Closes #2541 - TODO: Support the clienthello extension Closes #2299 - TODO: CLOEXEC Closes #2252 - tests: provide 'manual' as a feature to optionally require ... and make test 1026 rely on that feature so that --disable-manual builds don't cause test failures. Reported-by: Max Dymond and Anders Roxell Fixes #2533 Closes #2540 - CURLINFO_PROTOCOL.3: mention the existing defined names Jay Satiro (27 Apr 2018) - [Daniel Gustafsson brought this change] cookies: remove unused macro Commit 2bc230de63 made the macro MAX_COOKIE_LINE_TXT become unused, so remove as it's not part of the published API. Closes https://github.com/curl/curl/pull/2537 Daniel Stenberg (27 Apr 2018) - [Daniel Gustafsson brought this change] checksrc: force indentation of lines after an else This extends the INDENTATION case to also handle 'else' statements and require proper indentation on the following line. Also fixes the offending cases found in the codebase. Closes #2532 - http2: fix null pointer dereference in http2_connisdead This function can get called on a connection that isn't setup enough to have the 'recv_underlying' function pointer initialized so it would try to call the NULL pointer. Reported-by: Dario Weisser Follow-up to db1b2c7fe9b093f8 (never shipped in a release) Closes #2536 - http2: get rid of another strstr() Follow-up to 1514c44655e12e: replace another strstr() call done on a buffer that might not be zero terminated - with a memchr() call, even if we know the substring will be found. Assisted-by: Max Dymond Detected by OSS-Fuzz Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8021 Closes #2534 - cyassl: adapt to libraries without TLS 1.0 support built-in WolfSSL doesn't enable it by default anymore - configure: provide --with-wolfssl as an alias for --with-cyassl - RELEASE-NOTES: synced - [Daniel Gustafsson brought this change] os400.c: fix ASSIGNWITHINCONDITION checksrc warnings All occurrences of assignment within conditional expression in os400sys.c rewritten into two steps: first assignment and then the check on the success of the assignment. Also adjust related incorrect brace positions to match project indentation style. This was spurred by seeing "if((inp = input_token))", but while in there all warnings were fixed. There should be no functional change from these changes. Closes #2525 - [Daniel Gustafsson brought this change] cookies: ensure that we have cookies before writing jar The jar should be written iff there are cookies, so ensure that we still have cookies after expiration to avoid creating an empty file. Closes #2529 - strcpy_url: only %-encode values >= 0x80 OSS-Fuzz detected https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8000 Broke in dd7521bcc1b7 - mime: avoid NULL pointer dereference risk Coverity detected, CID 1435120 Closes #2527 - [Stephan Mühlstrasser brought this change] ctype: restore character classification for non-ASCII platforms With commit 4272a0b0fc49a1ac0ceab5c4a365c9f6ab8bf8e2 curl-speficic character classification macros and functions were introduced in curl_ctype.[ch] to avoid dependencies on the locale. This broke curl on non-ASCII, e.g. EBCDIC platforms. This change restores the previous set of character classification macros when CURL_DOES_CONVERSIONS is defined. Closes #2494 - ftplistparser: keep state between invokes Fixes FTP wildcard parsing when done over a number of read buffers. Regression from f786d1f14 Reported-by: wncboy on github Fixes #2445 Closes #2526 - examples/http2-upload: expand buffer to avoid silly warning http2-upload.c:135:44: error: ‘%02d’ directive output may be truncated writing between 2 and 11 bytes into a region of size between 8 and 17 - examples/sftpuploadresume: typecast fseek argument to long /docs/examples/sftpuploadresume.c:102:12: warning: conversion to 'long int' from 'curl_off_t {aka long long int}' may alter its value - Revert "ftplistparser: keep state between invokes" This reverts commit abbc8457d85aca74b7cfda1d394b0844932b2934. Caused fuzzer problems on travis not seen when this was a PR! - Curl_memchr: zero length input can't match Avoids undefined behavior. Reported-by: Geeknik Labs - ftplistparser: keep state between invokes Fixes FTP wildcard parsing when doing over a number of read buffers. Regression from f786d1f14 Reported-by: wncboy on github Fixes #2445 Closes #2519 - ftplistparser: renamed some members and variables ... to make them better spell out what they're for. - RELEASE-NOTES: synced - [Christian Schmitz brought this change] curl_global_sslset: always provide available backends Closes #2499 - http2: convert an assert to run-time check Fuzzing has proven we can reach code in on_frame_recv with status_code not having been set, so let's detect that in run-time (instead of with assert) and error error accordingly. (This should no longer happen with the latest nghttp2) Detected by OSS-Fuzz Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=7903 Closes #2514 - curl.1: clarify that options and URLs can be mixed Fixes #2515 Closes #2517 Jay Satiro (23 Apr 2018) - [Archangel_SDY brought this change] CURLOPT_SSLCERT.3: improve WinSSL-specific usage info Ref: https://github.com/curl/curl/pull/2376#issuecomment-381858780 Closes https://github.com/curl/curl/pull/2504 - [Archangel_SDY brought this change] schannel: fix build error on targets <= XP - Use CRYPT_STRING_HEX instead of CRYPT_STRING_HEXRAW since XP doesn't support the latter. Ref: https://github.com/curl/curl/pull/2376#issuecomment-382153668 Closes https://github.com/curl/curl/pull/2504 Daniel Stenberg (23 Apr 2018) - Revert "ftplistparser: keep state between invokes" This reverts commit 8fb78f9ddc6d858d630600059b8ad84a80892fd9. Unfortunately this fix introduces memory leaks I've not been able to fix in several days. Reverting this for now to get the leaks fixed. Jay Satiro (21 Apr 2018) - tool_help: clarify --max-time unit of time is seconds Before: -m, --max-time