webget
思路：
创建TCP套接字：使用系统调用如 socket() 创建一个TCP套接字。
设定服务器地址：根据服务器的IP地址和端口号构造地址结构。
连接服务器：使用 connect() 方法和服务器地址来建立TCP连接。
发送HTTP请求：构建合适的HTTP请求字符串。
使用套接字的文件描述符，通过 write() 或类似方法发送请求到服务器。
接收响应：循环使用 read() 从套接字读取数据到缓冲区，处理TCP的分片和粘包问题。
解析HTTP响应数据，按需处理（如保存数据、打印等）。
关闭连接：读取完所有数据后，使用 close() 关闭套接字，释放资源。需要注意的是：
（1）通常情况下，这里的 "service" 指的是端口号（如80用于HTTP，443用于HTTPS）
（2）HTTP头的每一行都应以 "\r\n" 结尾，并在整个请求的末尾加上一个额外的 "\r\n" 来结束请求。
（3）TCP是一个面向流的协议，EOF的判断通常是read函数返回0，表示对端关闭了连接
（4）文件描述符类似IO流的概念，文件描述符提供了一种抽象接口，用于在程序和操作系统之间传递文件和其他输入/输出设备的操作。
（5）在理想情况下，HTTP响应的所有数据应该能够一次性存储到足够大的缓冲区中。
（6）如果响应数据量较大，无法确定其大小，或者存在限制（如内存限制），则可能需要分批次地读取数据，即通过循环读取的方式。
（7）在循环读取数据的过程中，通过初始化变量（如read_offset = 0），读取一定数量的数据（如100字节），然后更新变量（如read_offset += 100），以确保后续读取操作能够从正确位置开始，以保证数据的连续性和完整性。
（8）当前文件描述符的行为已经隐含地处理了读取位置的跟踪（int fd_num()）实现了read_offset的功能。
最新代码:
void get_URL(const std::string& host, const std::string& path) {TCPSocket client;Address server(host, "http"); // Port should be 80 by default for HTTPclient.connect(server);// Properly formatted HTTP GET requestclient.write("GET " + path + " HTTP/1.1\r\n");client.write("Host: " + host + "\r\n");client.write("Connection: close\r\n\r\n"); // Note the double \r\nstd::string response;std::string buffer;while (!client.eof()) {buffer.clear(); // Clear the buffer to avoid accumulating old dataclient.read(buffer);response += buffer; // Append new data to the complete response}std::cout << response;client.close();
}测试结果:
ubun22@DESKTOP-1VEP92H:/mnt/e/Web/minnow$ cmake --build build --target check_webget
Test project /mnt/e/Web/minnow/buildStart 1: compile with bug-checkers
1/2 Test #1: compile with bug-checkers ........ Passed 4.88 secStart 2: t_webget
2/2 Test #2: t_webget ......................... Passed 1.66 sec100% tests passed, 0 tests failed out of 2Total Test time (real) = 6.69 sec
Built target check_webget
ubun22@DESKTOP-1VEP92H:/mnt/e/Web/minnow$ An in-memory reliable byte stream ：
注意：
（1）字节流close后，不需要把buffer置为空
（2）只有pop可能会影响position的值
（3）peek()就是直接返回buffer中从position到结尾的视图
最新代码：
byte_stream.cc:
bool Writer::is_closed() const
{return close_;
}void Writer::push( string data )
{if ( is_closed() ) {return;}uint64_t available_space = available_capacity();uint64_t data_size = data.size();uint64_t push_size = std::min( available_space, data_size );bytes_pushed_ += push_size;buffer_ += data.substr( position_, push_size );return;
}void Writer::close()
{close_ = true;return;
}uint64_t Writer::available_capacity() const
{return capacity_ - buffer_.size();
}uint64_t Writer::bytes_pushed() const
{return bytes_pushed_;
}bool Reader::is_finished() const
{if ( close_ && !buffer_.size() ) {return true;} else {return false;}
}uint64_t Reader::bytes_popped() const
{return bytes_popped_;
}std::string_view Reader::peek() const
{if ( position_ < buffer_.size() ) {// 如果缓冲区中有未读取的字节，则返回下一个字节的视图return std::string_view( &buffer_[position_], buffer_.size() - position_ );} else {// 如果缓冲区已经读取完毕，则返回一个空的 string_viewreturn std::string_view();}
}void Reader::pop( uint64_t len )
{if ( is_finished() ) {return;}uint64_t buffer_size = bytes_buffered();buffer_ = buffer_.substr( len, buffer_size );bytes_popped_ += len;if ( position_ >= len ) {position_ -= len;}return;
}uint64_t Reader::bytes_buffered() const
{return buffer_.size();
}byte_stream.hh:
#pragma once#include <cstdint>
#include <string>
#include <string_view>class Reader;
class Writer;class ByteStream
{
public:explicit ByteStream( uint64_t capacity );// Helper functions (provided) to access the ByteStream's Reader and Writer interfacesReader& reader();const Reader& reader() const;Writer& writer();const Writer& writer() const;void set_error() { error_ = true; }; // Signal that the stream suffered an error.bool has_error() const { return error_; }; // Has the stream had an error?protected:// Please add any additional state to the ByteStream here, and not to the Writer and Reader interfaces.uint64_t capacity_;std::string buffer_ = "";uint64_t position_ = 0;uint64_t bytes_popped_ = 0; // Initialize to zerouint64_t bytes_pushed_ = 0; // Initialize to zerobool close_ = false; // Initialize to falsebool error_ = false; // Initialize to false
};class Writer : public ByteStream
{
public:void push( std::string data ); // Push data to stream, but only as much as available capacity allows.void close(); // Signal that the stream has reached its ending. Nothing more will be written.bool is_closed() const; // Has the stream been closed?uint64_t available_capacity() const; // How many bytes can be pushed to the stream right now?uint64_t bytes_pushed() const; // Total number of bytes cumulatively pushed to the stream
};class Reader : public ByteStream
{
public:std::string_view peek() const; // Peek at the next bytes in the buffervoid pop( uint64_t len ); // Remove `len` bytes from the bufferbool is_finished() const; // Is the stream finished (closed and fully popped)?uint64_t bytes_buffered() const; // Number of bytes currently buffered (pushed and not popped)uint64_t bytes_popped() const; // Total number of bytes cumulatively popped from stream
};/** read: A (provided) helper function thats peeks and pops up to `len` bytes* from a ByteStream Reader into a string;*/
void read( Reader& reader, uint64_t len, std::string& out );
测试结果：
ubun22@DESKTOP-1VEP92H:/mnt/e/Web/minnow$ cmake --build build --target check0
Test project /mnt/e/Web/minnow/buildStart 1: compile with bug-checkers1/10 Test #1: compile with bug-checkers ........ Passed 2.38 secStart 2: t_webget2/10 Test #2: t_webget ......................... Passed 1.65 secStart 3: byte_stream_basics3/10 Test #3: byte_stream_basics ............... Passed 0.20 secStart 4: byte_stream_capacity4/10 Test #4: byte_stream_capacity ............. Passed 0.26 secStart 5: byte_stream_one_write5/10 Test #5: byte_stream_one_write ............ Passed 0.18 secStart 6: byte_stream_two_writes6/10 Test #6: byte_stream_two_writes ........... Passed 0.21 secStart 7: byte_stream_many_writes7/10 Test #7: byte_stream_many_writes .......... Passed 0.23 secStart 8: byte_stream_stress_test8/10 Test #8: byte_stream_stress_test .......... Passed 0.20 secStart 37: compile with optimization9/10 Test #37: compile with optimization ........ Passed 1.95 secStart 38: byte_stream_speed_testByteStream throughput: 1.02 Gbit/s
10/10 Test #38: byte_stream_speed_test ........... Passed 0.18 sec100% tests passed, 0 tests failed out of 10Total Test time (real) = 7.72 sec
Built target check0
