I believe I may have asked you this in the past, but I can’t find my notes: I believe you once mentioned having ported the complete OpenSSL stack to Net+OS, and offered to make this available at a price? Could you contact me about this at jwormsley at debitek dot com?
OpenSSL you can have for free. It is GNU. The issue it that it is TRUE OpenSSL, and the built in NetOS HTTPS will not interface with it, you need to compile your own SSL based code. Changes: I added arm-elf to the configurations; also NetOS is now recognized.
http://www.timewave.us/support/pub/openssl-0.9.7d.tar.gz
It is already all set up, just compile. It will stop when it hits the executables, but libssl.a and libcrypto.a will be complete at that point. Don’t forget to create a “openssl” in the “h” directory and copy over all the header files.
I ported it so that I could develop SSH, which is what I can provide at a cost (the code for that is about 10% of OpenSSH, 10% of FreeSSH, and 80% my code). If you are interested in this, contact me at egawtry at timewave dot com or timewave dot us.
-Erik
Thanks a ton! I’ll give this a shot as soon as I can. Hopefully I won’t find too many problems using it. I assume Net+OS stuff like NASSLX509Generate are out the window. I’ll probably also have to find sample applications on the web.
What I am trying to do is perform a SOAP over HTTPS transaction. I am the client side, so I don’t need the integration with Digi’s HTTPS code. But I do need to be able to authenticate the host, which Digi’s implementation won’t do. Of course, I’m still having trouble finding socket level examples of how to perform a SOAP transaction in the first place, but I’ll figure it out eventually. I’m currently trying to integrate libxml2 and csoap into the Digi, in which case I shouldn’t have too much trouble.
Thanks again!
Jeff.
I think I put the source in the wrong directory:
/cygdrive/c/netos63_gnu/src/openssl-0.9.7d
This causes the following error:
In file included from …/cryptlib.h:65,
from bss_fd.c:62:
…/…/e_os.h:413:32: tcpip/socket.h: No such file or directory
…/…/e_os.h:414:28: tcpip/in.h: No such file or directory
make[2]: *** [bss_fd.o] Error 1
make[2]: Leaving directory /cygdrive/c/netos63_gnu/src/openssl-0.9.7d/crypto/bi o' make[1]: *** [subdirs] Error 1 make[1]: Leaving directory /cygdrive/c/netos63_gnu/src/openssl-0.9.7d/crypto’
make: *** [sub_all] Error 1
Where should I place the source to compile?
Thanks,
Jeff.
I built it in netos63_gnu/src/apps/openssl-0.9.7d
I must have put a relative path in there. Oops.
-Erik
> I built it in netos63_gnu/src/apps/openssl-0.9.7d
>
> I must have put a relative path in there. Oops.
Hmmm, something else must be wrong there. I wiped out the copy under …/src and untarred the archive under …/src/apps and issued the following command:
make PLATFORM=connectme clean all
I get the same error message.
gcc -I… -I…/… -I…/…/include -DOPENSSL_THREADS -D_REENTRANT -DOPENSSL_NO_KRB
5 -D__GNU__ -DNETOS_GNU_TOOLS -DNET_OS -DNET_WORKS -DHAVE_RENAME -DNO_STRINGS_H
-DB_ENDIAN -D_POSIX_SOURCE -I/netos63_gnu/h -I/netos63_gnu/h/threadx -I/netos63_
gnu/h/tcpip -g -fomit-frame-pointer -Wall -mbig-endian -mcpu=arm7tdmi -c -o bs
s_fd.o bss_fd.c
In file included from …/cryptlib.h:65,
from bss_fd.c:62:
…/…/e_os.h:413:32: tcpip/socket.h: No such file or directory
…/…/e_os.h:414:28: tcpip/in.h: No such file or directory
make[2]: *** [bss_fd.o] Error 1
make[2]: Leaving directory /cygdrive/c/netos63_gnu/src/apps/openssl-0.9.7d/cryp to/bio' make[1]: *** [subdirs] Error 1 make[1]: Leaving directory /cygdrive/c/netos63_gnu/src/apps/openssl-0.9.7d/cryp
to’
make: *** [sub_all] Error 1
Clearly, the "-I/netos63_gnu/h/tcpip: is included on the command line. I wonder what clue this part gives:
“-I…/…/include”
There is no “include” directory two levels up from e_os.h, it is one directory below. I’ve also tried adjusting the -I to include the /cygdrive/c before the /netos63_gnu, but that seems to have no effect either.
I take that back, specifying /cygdrive/c does change things:
gcc -I… -I…/… -I…/…/include -DOPENSSL_THREADS -D_REENTRANT -DOPENSSL_NO_KRB
5 -D__GNU__ -DNETOS_GNU_TOOLS -DNET_OS -DNET_WORKS -DHAVE_RENAME -DNO_STRINGS_H
-DB_ENDIAN -D_POSIX_SOURCE -I/cygdrive/c/netos63_gnu/h -I/cygdrive/c/netos63_gnu
/h/threadx -I/cygdrive/c/netos63_gnu/h/tcpip -g -fomit-frame-pointer -Wall -mbig
-endian -mcpu=arm7tdmi -c -o bss_sock.o bss_sock.c
In file included from /cygdrive/c/netos63_gnu/h/sockapi.h:51,
from bss_sock.c:68:
/cygdrive/c/netos63_gnu/h/tcpip/ftype.h:60: error: redefinition of struct timev al' /cygdrive/c/netos63_gnu/h/tcpip/ftype.h:66: error: redefinition of struct timez
one’
bss_sock.c: In function sock_write': bss_sock.c:161: warning: passing arg 2 of send’ discards qualifiers from pointe
r target type
make[2]: *** [bss_sock.o] Error 1
make[2]: Leaving directory /cygdrive/c/netos63_gnu/src/apps/openssl-0.9.7d/cryp to/bio' make[1]: *** [subdirs] Error 1 make[1]: Leaving directory /cygdrive/c/netos63_gnu/src/apps/openssl-0.9.7d/cryp
to’
make: *** [sub_all] Error 1
JeffW@JEFFW /cygdrive/c/netos63_gnu/src/apps/openssl-0.9.7d
Modifying bss_sock.c to do this:
/*
#ifdef NETOS_GNU_TOOLS
#include
#endif
*/
allows compilation to continue, with lots of warnings about implicit use of functions… It’s still going, I’ll report back here if I figure out what’s wrong, or if this gets me through the problems.
You get all the function warnings even when compiling under Linux. It seems that some people have trouble with casting.
Except for minor changes in the setups I did not modify the files, so all those messages still appear.
Also, you may have to add this to your socket.h file:
#define AF_INET6 23 /* Internetwork Version 6 */
I don’t remember if I added it for SSL or SSH.
-Erik
I’ve not needed the AF_INET6 line, but I’ve ran into some other issues.
With the following changes, I can compile the library:
-
In bss_sock.c, comment out the addition you made to include sockapi.c (the reference to NETOS_GNU_TOOLS indicates you had to add it). If I leave this in, then I get a redefinition of timeval and timezone from ftype.h. I can’t find what is defining them beforehand, though. LDAP code has a definition that is said to avoid including headers, but as best as I can tell, LDAP isn’t being used.
-
In ssl_cert.c, comment out the inclusion of dirent.h. I can’t find that file anywhere on my system. I’m not sure how yours compiled without that change.
-
In Makefile, add “/cygdrive/c” to all paths starting with “/netos63_gnu”.
However, once this is done, I still cannot actually use the library. I run into errors at the very beginning. Using the openssl example apps from http://www.rtfm.com/openssl-examples/, I have tried to piece together what I have to do in my app since I don’t have any filesystem support. This means that every sample I’ve seen isn’t useable, because they all assume you are loading certificats, algorithm lists, etc from files. The best I’ve come up with as far as working past this starts out as follows:
#include
SSL_CTX * ctx;
SSL *ssl;
BIO *sbio;
SSL_METHOD *meth;
int sock;
struct sockaddr_in server;
SSL_library_init();
SSL_load_error_strings();
meth = SSLv23_method();
ctx = SSL_CTX_new(meth);
ssl = SSL_new(ctx);
sock = socket (AF_INET, SOCK_STREAM, 0);
server.sin_family = AF_INET;
server.sin_port = myParams.Host_Port;
server.sin_addr.s_addr = htonl(DNSgethostbyname((char *)&myParams.Host_Addr));
if (connect (sock, &server, sizeof(server)) >= 0)
{
sbio=BIO_new_socket(sock,BIO_NOCLOSE);
SSL_set_bio(ssl,sbio,sbio);
SSL_connect(ssl);
SSL_write(ssl,ReaderOutBuff.buf,ReaderOutBuff.buflen);
…
But, this fails, ostensibly at the “ssl = SSL_new(ctx);” line, causing an exception. However, digging deeper, the crash is caused by ctx returning zero (yes, no error checking in this test code), which in turn is caused by there being no registered crypto algorithms. This can be traced to the very first line of code, “SSL_library_init();”. Tracing through this code, first to EVP_add_cipher, then to OBJ_NAME_add, down into OBJ_nid2sn, shows that way down in obj_dat.c, there is the following code:
if ((n != NID_undef) && (nid_objs[n].nid == NID_undef))
{
OBJerr(OBJ_F_OBJ_NID2OBJ,OBJ_R_UNKNOWN_NID);
return(NULL);
}
The array nid_objs[n] is “empty”, so that for each n, nid_objs[n].nid == NID_undef. This array is initialized this way in obj_dat.h. So as each cipher is supposed to be added to the list, it looks like it is rejected because of the empty array, and therefore when it comes time to create the CTX, it fails because there are no registered cipher algorithms. This in turn stops SSL_new from working, and I get my crash.
I’m not really sure how all of this is supposed to work, and like I said, all of the documentation I have found assumes I’m working with a filesystem and a fully supported OS. I may have to take a turn trying to build this on Windows and following it’s initialization/call sequence to see what I can learn about how this is supposed to work. I really didn’t plan on going this low level, but sometimes one has no choice.
If you can spot where I have messed up, I’d appreciate a heads up, but it really isn’t your job to help me deal with this. I really appreciate all the help so far.
Thanks,
Jeff.
Ok, ignore a lot of the above…
The file obj_dat.h is not a “normal” source file. It is generated during make by calling a Perl script. This Perl script is supposed to take the contents of a file (according to the header in obj_dat.h, it is obj_mac.h, according to a Google search, it is objects.h, and somewhere, somehow there is a file called objects.txt that comes into play), and from that file generate obj_dat.h. However, what is generated is bogus.
#define NUM_NID 650
#define NUM_SN 0
#define NUM_LN 0
#define NUM_OBJ 0
static unsigned char lvalues[1]={
};
static ASN1_OBJECT nid_objs[NUM_NID]={
{NULL,NULL,NID_undef,0,NULL},
{NULL,NULL,NID_undef,0,NULL},
{NULL,NULL,NID_undef,0,NULL},
{NULL,NULL,NID_undef,0,NULL},
{NULL,NULL,NID_undef,0,NULL},
{NULL,NULL,NID_undef,0,NULL},
{NULL,NULL,NID_undef,0,NULL},
{NULL,NULL,NID_undef,0,NULL},
… 650 of these …
{NULL,NULL,NID_undef,0,NULL},
};
static ASN1_OBJECT *sn_objs[NUM_SN]={
};
static ASN1_OBJECT *ln_objs[NUM_LN]={
};
static ASN1_OBJECT *obj_objs[NUM_OBJ]={
};
The fact that all of these structures are empty is what causes the creation of the CTX to fail. There really are no algorithms defined, because they are all NULL/undefined.
I have no idea how to debug the Perl script. I could watch it execute with the Perl debugger, but the source all looks like an EXE file openened in Notepad.exe to me. I can’t make head nor tails what it is trying to do to figure out why it isn’t.
I also don’t understand how your SSH project works, since obj_dat.h is also messed up in your original tarball.
Jeff.
Update: Installed latest Win32 Perl, ran script, rebuilt and moved libraries, and I am now past the no algorithm problem. Now I am up against the following SSL error: 1:error:24064064:random number generator:SSLEAY_RAND_BYTES:PRNG not seeded:md_rand.c:503:You need to read the OpenSSL FAQ, http://www.openssl.org/support/faq.html
Adding a call to RAND_seed seems to get past this. I’m still locking up somewhere, but I think it is in my reception code. At any rate, I think I am getting very close now.
Thanks for all the help (even just being a sounding board helps!).
Jeff.
In the SSH code it calls
SSLeay_add_all_algorithms();
at the start.
As far as randomization goes, the following code is used by SSH:
#ifndef HAVE_ARC4RANDOM
/* Size of key to use */
#define SEED_SIZE 20
/* Number of bytes to reseed after */
#define REKEY_BYTES (1 << 24)
static int rc4_ready = 0;
static RC4_KEY rc4;
unsigned int arc4random(void)
{
unsigned int r = 0;
static int first_time = 1;
if (rc4_ready <= 0) {
if (first_time)
seed_rng();
first_time = 0;
arc4random_stir();
}
RC4(&rc4, sizeof(r), (unsigned char *)&r, (unsigned char *)&r);
rc4_ready -= sizeof(r);
return(r);
}
int arc4random_stir(void)
{
unsigned char rand_buf[SEED_SIZE];
memset(&rc4, 0, sizeof(rc4));
if (RAND_bytes(rand_buf, sizeof(rand_buf)) <= 0)
fatal("Couldn't obtain random bytes (error %ld)", ERR_get_error());
RC4_set_key(&rc4, sizeof(rand_buf), rand_buf);
RC4(&rc4, sizeof(rand_buf), rand_buf, rand_buf);
memset(rand_buf, 0, sizeof(rand_buf));
rc4_ready = REKEY_BYTES;
return 0;
}
#endif /* !HAVE_ARC4RANDOM */
#define RANDOM_SEED_SIZE 48
int
seed_rng(void)
{
unsigned char buf[RANDOM_SEED_SIZE];
// Note: See nisttime.cc for srand()
for(int i = 0; i < sizeof(buf); ++i )
{
buf[ i ] = rand()%0x100;
}
RAND_add(buf, sizeof(buf), sizeof(buf));
// memset(buf, 0, sizeof(buf)); // Extra security
for(int i = 0; i < sizeof(buf); ++i )
{
buf[ i ] = rand()%0x100;
}
RAND_add(buf, sizeof(buf), sizeof(buf));
// memset(buf, 0, sizeof(buf)); // Extra security
for(int i = 0; i < sizeof(buf); ++i )
{
buf[ i ] = rand()%0x100;
}
RAND_add(buf, sizeof(buf), sizeof(buf));
// memset(buf, 0, sizeof(buf)); // Extra security
if (RAND_status() != 1)
fatal("PRNG is not seeded");
return 0;
}
int
init_rng(void)
{
/*
* OpenSSL version numbers: MNNFFPPS: major minor fix patch status
* We match major, minor, fix and status (not patch)
*/
if ((SSLeay() ^ OPENSSL_VERSION_NUMBER) & ~0xff0L)
fatal("OpenSSL version mismatch. Built against %lx, you "
“have %lx”, OPENSSL_VERSION_NUMBER, SSLeay());
return 0;
}
-Erik
Message was edited by: egawtry because of messed up bracketing by forum.
The RAND_seed() call was enough to move forward. Since the seed is fixed, it probably isn’t a production solution, though.
My lockup mentioned above was indeed in my read. Since I don’t know the size of an incoming packet, my loop would keep calling read until a timeout, which was some huge amount of time. I got around this by setting the initial socket to blocking and adding SSL_CTX_set_mode(ctx, SSL_MODE_AUTO_RETRY) to my initialization. This means when the SSL_read returns, it has all of the data I’m going to get, and I don’t have to worry about renegotiations or anything else.
I am now happily trying to decode XML/SOAP responses to http queries over SSL! Oh frabjuous day!
Jeff.