From 9933cec0a18ae2a3d752f269d1bb12c19f51199d Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sun, 21 Jul 2013 15:35:02 -0400 Subject: IRC. --- community/gsoc/2013/hacklu.mdwn | 617 ++++++++++++++++++++++++++++++++++ community/gsoc/2013/nlightnfotis.mdwn | 450 +++++++++++++++++++++++++ 2 files changed, 1067 insertions(+) create mode 100644 community/gsoc/2013/hacklu.mdwn create mode 100644 community/gsoc/2013/nlightnfotis.mdwn (limited to 'community/gsoc/2013') diff --git a/community/gsoc/2013/hacklu.mdwn b/community/gsoc/2013/hacklu.mdwn new file mode 100644 index 00000000..d0185c60 --- /dev/null +++ b/community/gsoc/2013/hacklu.mdwn @@ -0,0 +1,617 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!toc]] + + +# IRC, freenode, #hurd, 2013-06-23 + + braunr: sorry for the late reply. Honestly to say, the school + works had taken most of my time these days. I haven't got any + siginificant progress now. I am trying to write a little debugger demo on + Hurd. + braunr: things goes more hard than I think, these are some + differences between ptrace() on Hurd and Linux. I am trying to solve + this. + + +# IRC, freenode, #hurd, 2013-06-24 + + this is my weekly report + http://hacklu.com/blog/gsoc-weekly-report1-117/. + and I have two main questions when I read the gdb source code. + 1/What is the S_exception_raise_request()? 2/what is the role of + ptrace in gdb port on Hurd? + hacklu: where did you see S_exception_raise_request? + in gdb/gnu-nat.c + ah, in gdb + yeah. and I have read the . is says the S_ + start means server stub. + yes + what happens is that gnu_wait keeps calling mach_msg + to get a message + then it passes that message to the various stubs servers + see just below, it calls exc_server, among others + and that's exc_server which ends up calling + S_exception_raise_request, if the message is an exception_raise request + exc_server is a mere multiplexer, actually + S_exception_raise_request is the implementation of the request + part (so one half of a typical RPC) of the Mach exception interface. + See gdb/exc_request.defs in GDB and include/mach/exc.defs in + Mach. + youpi: how gnu_wait pass one message to exc_server? in which + function? + in gnu_wait() + && !exc_server (&msg.hdr, &reply.hdr) + oh, I see this. + firstly I think it is a type check simply. + see the comment: "handle what we got" + The Hurd's proc server also is involved in the exception + passing protocol (see its source code). + tschwinge: I will check the source code later. is the exception + take place in this way: 1. the inferior call ptrace(TRACE_ME). 2.the gdb + call task_set_exception_port. 3. mach send a notification to the + exception port set before. 4. gdb take some action. + hacklu: Yes, that's it, roughly. The idea is that GDB replaces + a process' standard exception port, and replaces it "with itself", so + that when the process that is debugged receives and exception (by Mach + sending a exception_raise RPC), GDB can then catch that and act + accordingly. + hacklu: As for your other questions, about ptrace: As you can + see in [glibc]/sysdeps/mach/hurd/ptrace.c, ptrace on Hurd is simply a + wrapper around vm_read/write and more interfaces. + hacklu: As the GDB port for Hurd is specific to Hurd by + definition, you can also directly use these calls in GDB for Hurd. + ..., as it is currently done. + and in detail, the part 3 mach send a notification to the + excetption port is like this: gnu_wait get the message in mach_msg, and + then pass it to exc_serer by exc_server(),then exc_server call + S_exception_raise_request()? ? + tschwinge: yeah, I have see the ptrace.c. I was wonder about + nobody use ptrace in Hurd except TRACEME... + hacklu: Right about »and in detail, [...]«. + hacklu: It would be very good (and required for your + understanding anyway), if you could write up a list of things that + happens when a process (both under the control of GDB as well as without + GDB) is sent an exception (due to a breakpoint instruction, for example). + Let me look something up. + tschwinge: what's the function of exc_server? if I can get the + notification in mach_msg(). + to multiplex the message + i.e. decoding it, etc. up to calling the S_ function with the + proper parameters + exc_server being automatically generated, that saves a lot of code + That is generated by MIG from the gdb/exc_request.defs file. + You'll find the generated file in the GDB build directory. + I have wrote down the filenames. after this I will check that. + hacklu: I suggest you also have a look at the Mach 3 Kernel + Principles book, + . + This also has some explanation of the thread/task's exception + mechanism. + And of course, explains the RPC mechanism, which the exception + mechanism is built upon. + And then, really make a step-by-step list of what happens; this + should help to better visualize what's going on. + ok. later I will update this list on my blog. + hacklu: I cannot tell off-hand why GDB on Hurd is using + ptrace(PTRACE_TRACEME) instead of doing these calls manually. I will + have to look that up, too. + tschwinge: thanks. + hacklu: Anyway -- you're asking sensible questions, so it seems + you're making progress/are on track. :-) + tschwinge: there is something harder than I had thought, I haven't + got any meaningful progress. sorry for this. + hacklu: That is fine, and was expected. :-) (Also, you're + still busy with university.) + I will show more time and enthusiasm on this. + hacklu: Oh, and one thing that may be confusing: as you may + have noticed, the names of the same RPC functions are sometimes slightly + different if different *.defs files. What is important is the subsystem + number, such as 2400 in [GDB]/gdb/exc_request.defs (and then incremented + per each routine/simpleroutine/skip directive). + hacklu: Just for completeness, [hurd]/hurd/subsystems has a + list of RPC subsystems we're using. + And the name given to routine 2400, for example, is just a + "friendly name" that is then used locally in the code where the *.defs + file has been processed by MIG. + What a clumsy explanation of mine. But you'll get the idea, I + think. ;-) + hacklu: And don't worry about your progress -- you're making a + lot of progress already (even if it doesn't look like it, because you're + not writing code), but the time spent on understanding these complex + issues (such as the RPC mechanism) definitely counts as progress, too. + tschwinge: not clearly to got it as I am not sensitive with the + MIG's grammer. But I know, the exc is the routine 2400's alias name? + hacklu: I'd like to have you spend enough time to understand + these fundamental concepts now, and then switch to "hacking mode" (write + code) later, instead of now writing code but not understanding the + concepts behind it. + I have wrote a bit code to validate my understanding when I read + the soruce code. But the code not run. http://pastebin.com/r3wC5hUp + The subsystem directive [...]. As well, let me just point you + to the documentation: + , + MIG - THE MACH INTERFACE GENERATOR, chapter 2.1 Subsystem identification. + hacklu: Yes, writing such code for testing also is a good + approach. I will have to look at that in more detail, too. + * tschwinge guesses hacklu is probably laughing when seeing the years these + documents were written in (1989, etc.). ;-) + mach_msg make no sense in my code, and the process just hang. kill + -9 can't stop is either. + hacklu: do you understand why kill -KILL might not work now ? + braunr: no, but I found I can use gdb to attach to that process, + then quit in gdb, the process quit too. + maybe that process was waiting a resume. + something like that yes + iirc it's related to a small design choice in the proc server + something that put processes in an uninterruptible state when + being debugged + iirc ? + if i recall cl=orrectly + correctly* + like D status in linux? + or T + there has been a lot of improvements regarding signal handling in + linux over time so it's not really comparable now + but that's the idea + in ps, i see the process STAT is THumx + did you see that every process on the hurd has at least two + threads ? + no, but I have see that in hurd, the exception handler can't live + in the same context with the victim. so there must be at least two + threads. I think + hacklu: yes + that thread also handles regular signals + in addition to mach exceptions + (there are two levels of multiplexing in servers, first locating + the subsystem, then the server function) + hacklu: if what i wrote is confusing, don't hesitate to ask for + clarifications (i really don't intend to make things confusing) + braunr: I don't know what you say about the "multiplexing in + servers". For instance, is it means how to pass message from mach_msg to + exc_server in gnu_wait()? + hacklu: i said that the "message thread" handles both mach + exceptions and unix signals + hacklu: these are two different interfaces (and in mach terms, + subsystems) + hacklu: see hurd/msg.defs for the msg interface (which handles + unix signals) + hacklu: to handle multiple interfaces in the same thread, servers + need to first find the right subsystem + this is done by subsequently calling all demux functions until one + returns true + (finding the right server function is done by these demux + functions) + hacklu: see hurd/msgportdemux.c in glibc to see how it's done + there + it's short actually, i'll past it here : + return (_S_exc_server (inp, outp) || + _S_msg_server (inp, outp)); + hacklu: did that help ? + braunr: a bit more confusing. one "message thread" handles + exceptions and signals, means the message thread need to recive message + from two port. then pass the message to the right server which handle the + message. the server also should pick the right subsystem from a lot of + subsystems to handle the msg. is this ? + the message thread is a server thread + (which means every normal process is actually also a server, + receiving exceptions and signals) + there may be only two ports, or more, it doesn't matter much, the + port set abstraction takes care of that + so the message thread directly pass the msg to the right + subsystem? + not directly as you can see + it tries them all until one is able to handle the incoming message + i'm not sure it will help you with gdb, but it's important to + understand for a better general understanding of the system + ugly sentence + ah, I see. like this in gnu-nat.c if(!notify_server(&msg.hdr, + &reply.hdr) && !exc_server(&msg.hdr...) + yes + the thread just ask one by one. + be careful about the wording + the thread doesn't "send requests" + it runs functions + (one might be tempted to think there are other worker threads + waiting for a "main thread" to handle demultiplexing messages) + I got it. + the notify_server function is just run in the same context in + "message thread",and there is no RPC here. + yes + and the notify_server code is generater by mig automatically. + yes + + +# IRC, freenode, #hurd, 2013-06-29 + +[[!tag open_issue_documentation]] + + I just failed to build the demo on + this. http://walfield.org/pub/people/neal/papers/hurd-misc/ipc-hello.c + or, example in machsys.doc called simp_ipc.c + we don't use cthreads anymore, but pthreads + pinotree: em.. and I also failed to find the + in example of + that i don't know + maybe the code in that book out-of-date + hacklu: mig and mach ipc documentation is quite dated + unfortunately, and so are many examples floating around the net + + btw, I have one more question. when I read . I find this state: When an exception occurs in a thread, the + thread sends an exception message to + its exception port, blocking in the kernel waiting for the receipt + of a reply. It is + assumed that some task is listening to this + port, using the exc_serverfunction to decode the messages and + then call the + linked in catch_exception_raise. It is the job of + catch_exception_raiseto handle the exception and decide the course of + action for thread. + that says, it assumed another task to recieve the msg send to one + thread's exception port. why another task? + I remmebered, there are at least two threads in one task, one is + handle the exception stuffs. + there are various reasons + first is, the thread causing the exception is usually not waiting + for a message + next, it probably doesn't have all the info needed to resolve the + exception + (depending on the system design) + and yes, the second thread in every hurd process is the msg + thread, handling both mach exceptions and hurd signals + but in this state, I can't find any thing with the so called msg + thread + ? + if exist a task to do the work, why we need this thread? + this thread is the "task" + ? + the msg thread is the thread handling exceptions for the other + threads in one task + wording is important here + a task is a collection of resources + so i'm only talking about threads really + 14:11 < hacklu> assumed that some task is listening to this + this is wrong + a task can't listen + only a thread can + in you words, the two thread is in the same task? + yes + 14:32 < braunr> and yes, the second thread in every hurd process + is the msg thread, handling both mach exceptions and hurd signals + process == task here + yeah, I always think the two thread stay in one task. but I found + that state in . so I confuzed + s/confuzed/confused + statement you mean + if two thread stay in the same task. and the main thread throw a + exception, the other thread to handle it? + depends on how it's configured + the thread receiving the exceptions might not be in the same task + at all + on the hurd, only the second thread of a task receives exception + s + I just wonder how can the second thread catch the exception from + its containning task + forget about tasks + tasks are resource containers + they don't generate or catch exceptions + only threads do + for each thread, there is an exception port + that is, one receive right, and potentially many send rights + the kernel uses a send right to send exceptions + the msg thread waits for messages on the receive right + that's all + ok. if I divide zero in main thread, the kernel will send a msg to + the main thread's exception port. and then, the second thread(in the same + task) is waiting on that port. so he get the msg. is it right? + don't focus on main versus msg thread + it applies to all other threads + as well + otherwise, you're right + ok, just s/main/first + no + main *and* all others except msg + main *and* all others except msg ? + the msg thread gets exception messages for all other threads in + its task + (at least, that's how the hurd configures things) + got it. + if the msg thread throw exception either, who server for himself? + i'm not sure but i guess it's simply forbidden + i used gdb to attach a little progrom which just contains a divide + zero. and I only found the msg thread is in the glibc. + yes + where is the msg thread located in. + it's created by glibc + is it glibc/hurd/catch-exc.c? + that's the exception handling code, yes + there are some differences between the code and the state in . + state or statement ? + staement + which one ? + http://pastebin.com/ZTBrUAsV + When an exception occurs in a thread, the thread sends an exception + message to + its exception port, blocking in the kernel waiting for the receipt of a + reply. It is + assumed that some task is listening (most likely with mach_msg_server) + to this + port, using the exc_serverfunction to decode the messages and then + call the + linked in catch_exception_raise. It is the job of + catch_exception_raiseto handle the exception and decide the course of + action for thread. The state of the + blocked thread can be examined with thread_get_state. + what difference ? + in the code, I can't find things like exc_server,mach_msg_server + uh + ok it's a little tangled + but not that much + you found the exception handling code, and now you're looking for + what calls it + simple + see _hurdsig_fault_init + from that statemnet I thought there are another _task_ do the + exception things for all of the systems thread before you have told me + the task means the msg thread. + again + 14:47 < braunr> forget about tasks + 14:47 < braunr> tasks are resource containers + 14:47 < braunr> they don't generate or catch exceptions + 14:47 < braunr> only threads do + yeah, I think that document need update. + no + it's a common misnomer + once you're used to mach concepts, the statement is obvious + braunr: so I need read more :) + _hurdsig_fault_init send exceptions for the signal thread to the + proc server? + why come about _proc_ server? + no it gives the proc server a send right for signals + exceptions are a mach thing, signals are a hurd thing + the important part is + err = __thread_set_special_port (_hurd_msgport_thread, + THREAD_EXCEPTION_PORT, sigexc); + this one set the exception port? + yes + hm wait + actually no, wrong part :) + this sets the excpetion port for the msg thread (which i will call + the signal thread as mentioned in glibc) + but the comment above this line, Direct signal thread exceptions + to the proc server means what? + that the proc server handles exceptions on the signal thread + the term signal thread equals the term msg thread? + yes + so, the proc server handles the exceptions throwed by the msg + thread? + looks that way + feels a little strange. + why ? + this thread isn't supposed to cause exceptions + if it does, something is deeply wrong, and something must clean + that task up + and the proc server seems to be the most appropriate place from + where to do it + why need a special server to just work the msg thread? I don't + think that thread will throw exception frequentlly + what does frequency have to do with anything here ? + ok the appropriate code is _hurdsig_init + the port for receiving exceptions is _hurd_msgport + the body of the signal thread is _hurd_msgport_receive + aha, in the _hurd_msgport_receive I have finally found the + while(1) loop mach_msg_server(). + so the code is conform with the documents. + braunr: [21:18] what does frequency have to do with + anything here ? yes, I have totally understood your words now. thank you + very much. + :) + + +# IRC, freenode, #hurd, 2013-07-01 + + hi. this is my weekly + report. http://hacklu.com/blog/gsoc-weekly-report2-124/ welcome to any + comment + teythoon: I only get clear about the rpc stuff. seems a lot behind + my plan + good progress :) + I have wrote the details of the exception handle which was asked + by tschwing_ last week. Am I all right in my post? + hacklu: as far as I understand signals, yes :) + youpi: thanks for god, I am on the right way finally... :) + the mig book says simpleroutine is the one use to implement asyn + RPCs which doesn't expect an reply. But I have found a place to pass an + reply port to the RPC interface which has been declared as simpleroutine + hacklu: probably the simpleroutine hardcodes a reply port? + + hacklu: about _hurd_internal_post_signal, this is the hairiest part + of GNU/Hurd, signal handling + simply because it's the hairiest part of POSIX :) + you probably want to just understand that it implements the + POSIXity of signal delivering + i.e. deliver/kill/suspend the process as appropriate + I don't think you'll need to dive more + aha. + it will save a lot of time. + it seems like the wait_for_inferior() in gdb. which also has too + many lines and too many goto + hacklu: btw, which simpleroutine were you talking about ? + I forget where it is, I am finding it now. + which version of gdb are you looking the source of? + (in mine, wait_for_inferior is only 45 lines long) + I dont know how to pick the verison, I just use the git + version. maybe I give a wrong name. + ok + youpi:I remembered, my experience comes from here + http://www.aosabook.org/en/gdb.html. (All of this activity is managed by + wait_for_inferior. Originally this was a simple loop, waiting for the + target to stop and then deciding what to do about it, but as ports to + various systems needed special handling, it grew to a thousand lines, + with goto statements criss-crossing it for poorly understood + reasons.) + youpi: the simpleroutine is gdb/gdb/exc_request.defs + so there is indeed an explicit reply port + but simpleroutine is for no-reply use. why use reply port here? + AIUI, it's simply a way to make the request asynchronous, but still + permit an answer + ok, I will read the mig book carefully. + hacklu: as youpi says + a routine can be broken into two simpleroutines + that's why some interfaces have interface.defs, + interface_request.defs and interface_reply.defs files + nlightnfotis: in mach terminology, a right *is* a capability + the only thing mach doesn't easily provide is a way to revoke them + individually + braunr: Right. And ports are associated with the process + server and the kernel right? I mean, from what I have understood, if a + process wants to send a signal to another one, it has to do so via the + ports to that process held by the process server + and it has to establish its identity before doing so, so + that it can be checked if it has the right to send to that port. + yes + do process own any ports? or are all their ports associated + with the process server? + *processes + mach ports were intended for a lot of different uses + but in the hurd, they mostly act as object references + the process owning the receive right (one at most per port) + implements the object + processes owning send rights invoke methods on the object + use portinfo to find out about the rights in a task + (process is the unix terminology, task is the mach terminologyà + ) + i use them almost interchangeably + ahh yes, I remember about the last bit. And mach tasks have + a 1 to 1 association with user level processes (the ones associated with + the process server) + the proc server is a bit special because it has to know about all + processes + yes + +In context of [[open_issues/libpthread/t/fix_have_kernel_resources]]: + + hacklu: if you ever find out about either glibc or the proc server + creating one receive right for each thread, please let me know + + +# IRC, freenode, #hurd, 2013-07-07 + + how fork() goes? + see sysdeps/mach/hurd/fork.c in glibc' sources + when the father has two thread( main thread and the signal thead), + if the father call fork, then the child inmediatelly call exev() to + change the excute file. how many thread in the children? + For instance, the new execute file also have two thread. + will the exev() destroyed two threads and then create two new? + s/exev()/excv() + s/exev()/exec() :) + + what libhurduser-2.13.so does? + where can I find this source? + contains all the client stubs for hurd-specific RPCs + it is generated and built automatically within the glibc build + process + + and what is the "proc" server? + what handles in user spaces the processes + so if I call proc_wait_request(), I will go into the + S_proc_wait_reply? + thanks, I have found that. + + +# IRC, freenode, #hurd, 2013-07-08 + + hi, this is my weekly + report. http://hacklu.com/blog/gsoc-weekly-report3-137/ + this week I have met a lot of obstacles. And I am quite desired to + participate in this meeting. + hacklu: So from your report, the short version is: you've been + able to figure out how the things work that you were looking at (good!), + and now there are some new open questions that you're working on now. + hacklu: That sounds good. We can of course try to help with + your open questions, if you're stuck figuring them out on your own. + tschwinge: the most question is: what is the proc server? why need + to call proc_get_reqeust() before the mach_msg()? + and Is there exist any specific running sequence between father + and child task after fork()? And I found the inferior always call the + trace_me() in the same time(the trace me printf always in the same line + of the output log). which I have post in my report. + hacklu: The fork man-page can provide a high-level answer to + your Q3: »The child process is created with a single thread—the one that + called fork(). The entire virtual address space of the parent is + replicated in the child, including the states of mutexes, condition + variables, and other pthreads objects [...]« + hacklu: What happens in GNU Hurd is that the signal thread is + also "cloned" (additionally to the thread which called fork), but then it + (the signal thread) is re-started from the beginning. (So this is very + much equivalent to creating a new signal thread.) + hacklu: Then, upon exec, a new memory image is created/loaded, + replacing the previous one. [glibc]/sysdeps/mach/hurd/execve.c. What + actually happens with the existing thread (in particular, the signal + thread) I don't know off-hand. Then answer is probably found in + [glibc]/hurd/hurdexec.c -- and perhaps some code of the exec server + ([hurd]/exec/). + I have checked the status of my regiter mail to FSF. it says it + had arrived in USA. + hacklu: OK, good. + hacklu: This is some basic information about the observer_* + functions is GDB: + http://sourceware.org/gdb/current/onlinedocs/gdbint/Algorithms.html#index-notifications-about-changes-in-internals-57 + »3.10 Observing changes in gdb internals«. + tschwinge: not too clear. I will think this latter. and what is + the proc server? + hacklu: /hurd/proc, maps unix processes to mach threads afaiui + teythoon: question is, the mach_msg() will never return unless I + called proc_wait_request() first. + hacklu: sorry, I've no idea ;) + teythoon: :) + hacklu: I will have to look into that myself, too; don't know + the answer off-hand. + hacklu: In your blog you write proc_get_request -- but such a + functions doesn't seems to exist? + tschwinge: s/proc_get_request/proc_wait_request called in + gun_wait() [gnu-nat.c] + hacklu: Perhaps the wait man-page's description of WUNTRACED + gives a clue: »also return if a child has stopped [...]«. But it also to + me is not yet clear, how this relates to the mach_mag call, and how the + proc server exactly is involved in it. + I'm reading various source code files. + At least, I don't undestand why it is required for an exception + to be forwarded. + if I need to read the proc server source code? + I can see how it to become relevant for the case that GDB has + to be informed that the debugee has exited normally. + hacklu: Yeah, probably you should spend some time with that, as + it will likely help to get a clearer picture of the situation, and is + relevant for other interactions in GDB, too. + hacklu: By the way, if you find that pieces of the GDB source + code (especially the Hurd files of it) are insufficiently documented, + it's a very good idea, once you have figured out something, to add more + source code comments to the existing code. Or writed these down + separately, if that is easier. + which is the proc server? hurd/exec ? + that ok, I already comment things on my notes. + hacklu: [Hurd]/proc/ + hacklu: And [Hurd]/hurd/process*.defs + got it + hacklu: I'll have to experiment a bit with your HDebugger + example, but I'm out of time right now, sorry. Will continue later. + tschwinge: yep, the HDebugger has a problem, if you put the + sleep() after the printf in the just_print(), thing will hang. + tschwinge: and I am a little curious about how do you find my + code? I dont't remember I have mentioned that :) + tschwinge: I have post my gihub link in the last week report, I + found that. + hacklu: That's how I found it, yes. + tschwinge: :) diff --git a/community/gsoc/2013/nlightnfotis.mdwn b/community/gsoc/2013/nlightnfotis.mdwn new file mode 100644 index 00000000..43f9b14c --- /dev/null +++ b/community/gsoc/2013/nlightnfotis.mdwn @@ -0,0 +1,450 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!toc]] + + +# IRC, freenode, #hurd, 2013-06-29 + + so, how is your golang port going? + I just started working on it. I had been reading + documentation so far. Maybe over reading as people told me when I asked + for their feedback + but I will report on what I have done (technically tomorrow, + and post it in the mailing list too. + + Hey guys, what could possibly cause the following error + message when executing a program in the Hurd? "./dumper: Could not open + note: (system server) error with unknown subsystem" + My program is one that opens a file and dumps it into stdout + pinotree: the code I am using is the one present here + http://www.gnu.org/software/hurd/hacking-guide/hhg.html under paragraph + 6.1 + I investigated it a bit but can not find a lead. I seem to + have all the rights to open the file that I want to dump to stdout + what if you reset errno to 0 just after all the declarations in + main, before the instructions? + will check this out and get back to you. + sure :) + pinotree: Now it suggests that it can't get the number of + readable files, which the source suggests that is normal behavior. + Thanks for your assistance. + + +# IRC, freenode, #hurd, 2013-07-01 + + youpi: from my part I can report that I have started working + with the code, and doing as Thomas suggested. I was about to write my + report yesterday, but I am facing some build errors on the HURD, which I + would like to investigate further before I write my report. + that's why I decided to write it later in the day. + I don't think you have to wait + you can simply write in your report that you are having build + errors + ok. I will have it written and delivered later in the day. + braunr: that's cool. I think my reading has paid for + itself. And you may be pleased to know that I have gotten my hands dirty + with the code. I was about to write report yesterday, but some build + errors with the gcc (that I am investigating atm) are holding me + off. Will have that written later in the day. + don't hesitate to ask help about build errors + don't wait too much + you need to progress on what matters, and not be blocked by + secondary problems + I will see myself asking for help rather sooner than later, + but I would like to investigate it myself, and attempt to solve the + issues that occur to me before resort to bugging you guys. + sure + just not too long + too long being a day or so + these were my build_results on the hurd + they were linker errors + + https://gist.github.com/NlightNFotis/5896188#file-build_results + I am trying to build gcc on a linux 32 bit environment. It + also has some issues but not linker errors + will resolve them to see if the linker errors are + reproducible on linux + oh, lex stuff + should be easy enough + + +# IRC, freenode, #hurd, 2013-07-05 + + I have not made much progress, but I see myself working with + it. + I have managed to build gcc go on Linux + but Hurd seems to have some issues + it seems to randomly crash + the build process? + not quite randomly it seems to be though + yeah + I have noticed that there is a pattern + it does crash after some time + ^^ + but it doesn't crash at specific files + define crash + at some times it may crash during compiling insn-emit.c + (hello guys) + hi braunr :) + braunr: hey there! It does seem to keep on compiling this + file for a very long time (I have let it do so for 10, 20, 30 minutes) + but the result is the same + and it does so for different files for different build + options + ok so it doesn't crash + it just doesn't complete + is the virtual machine eating 100% cpu during that time ? + I can still type at the terminal, but I can't send a term + signal + I can report that QEMU does hold 100% of one core at that + time, (like it keeps processing) but there is no output on the terminal + ok + of course I can type at the terminal + but nothing happens + any idea of the size of the files involved ? + I am checking it out right now + before this goes any further, let me report on my + investigation + i expect that to be our classic writeback thread storm issue + initially, I thought it might be that it run out of memory + even though I know that compilation is not memory intensive, + rather, cpu intensive + anyway I increased the size of ram available to the vm + from 1024 mb to 1536 + that didn't seem to have any effect. The "crash" still + happens at the same time, at the same files + use freeze + not crash + crash is very misleading here + freeze it is then. + anyway + then it striked me that it might be that the hard disk size + (3gb) might be too small (considering the gcc git repo is 1gb+) + so I resized the qemu image to 8gb of hdd size + the new size is acknowledged by the vm + for gcc in debug mode? might still not be enough + but still it has no effect - it seems to follow its freezing + patterns + giving your work, i'd have not less than 15-20 + i'd use 32 + *given + but that's because i like power of twos + pinotree: thanks for the advice. Right now I was gonna + increase the swap size + according to vmstat in the hurd + swap size is 173 mb + don't know if it does have an impact + it may but before rushing + if you need swap, you're doomed anyway + consider swap highly unreliable on the hurd + please show the output of df -h on the file system you're using to + build + ideally, i'd recommend using separate / and /home file systems + it really improves reliability + I don't think it swaps to be honest; however that's + something that my mentor thomas had suggested (increasing swap size) so I + am gonna try it at some time. + or have a separate file system in a subdi and work on it + yes, /home or whatever suits you + just not / + braunr: pinotree: thanks both for your advice. Will do now, + and report on the results. + that's not all + 11:17 < braunr> please show the output of df -h on the file system + you're using to build + braunr: I am on it. Oh and btw, everytime I am forced to + close the vm (due to the freezes) when I restart it ext2 reports that the + file system was not cleanly unmounted and does some repair to some + files. I am trying to find an explanation for that, but I can think of + many things + well obviously + ext2 has no journaling + the file system was not cleanly unmounted since you restarted it + with a cold reset + braunr: df -h comes out with this: "df: cannot read table of + mounted file systems" + also, even if you manage to always shut down correctly, when + fsck runs because of the maximum mount count it'd find errors anyway (so + we have some bug) + nlightnfotis: df -h /path/to/build/dir + pinotree: not really bugs but it could be cleaned up + filesystem: - Size 2.8G Used 2.8G Avail 0 Use% 100% Mounted + on / + wow + nlightnfotis: see + that seems to explain many things + ^^ + thanks for that braunr! + you resized the disk, but not the partition and the file system + braunr: well, if something in ext2 (or its libs) leaves issues + in the fs, i'd call that a bug :> + yeah, that was utterly stupid of me + pinotree: they're not issues + nlightnfotis: be careful, mach needs a reboot every time you + change a partition table + nlightnfotis: important thing is that you found the issue :) + then only, you can use resize2fs + braunr: weird, I thought mach nowadays can reload the partition + tables? + braunr: doesn't d-i need that? + maybe a recent change i forgot + or maybe fdisk still reports the error although it's fine + in doubt, rebooting is still safe :p + or maybe youpi hacked it into d-is gnumach + i doubt it would be there for the installer only :) + if it's there, it's there + i just don't know it + braunr: teythoon: and everyone else that helped me. Thanks + you all guys. This was something that was driving me crazy. Will do all + that you suggested and report back on my status + + +# IRC, freenode, #hurd, 2013-07-08 + + tschwinge, I have managed to overcome most of the obstacles + I had initially faced with my project + but I still had some build errors, that's why I have not + reported yet. Wanna try to see if I can resolve them today, and write my + report in the afternoon. + nlightnfotis: So, from a quick look into the IRC backlog, it + was a "simple" out of disk space problem? %-) That happens. + nlightnfotis: And yes, GCC needs a lot of disk space. + nlightnfotis: What kind of build errors are you seeing now? + tschwinge, yeah I felt stupid at the time, but it didn't + actually strike me that the file system didn't see the extra space. Also + it took me some time to figure out that in order to mount the new + partition, I only had to edit /etc/fstab + always tried to mount it with the ext2 translator + and the translator kept dying + but it's all figured out now + the latest build errors I am seeing are these + nlightnfotis: o_O you used fstab and it worked? + yeah + nlightnfotis: that's unexpected from my perspective... + I only had to add the new partition into fstab + teythoon: I can pastebin my fstab if you wanna take a look + at it + tschwinge: these were my latest build errors + https://www.dropbox.com/s/b0pssdnfa22ajbp/build_results + nlightnfotis: I'm pretty sure that mount -a isn't done on hurd + w/o pinos runsystem.sysv + weird + tschwinge: I have also tried to build gcc with "make -w" + which from what I know supresses the errors that stopped compilation + but the weird thing is that gcc nearly took forever to build + nlightnfotis: could you do a showtrans /your/mountpoint? + teythoon: /hurd/ext2fs /dev/hd0s3 + nlightnfotis: ok, so you've set a passive translator and an + active is started on demand + it must be a passive translator + nlightnfotis: this is the hurd way of doing things, fstab is + unrelated + it seems to persist during reboots + yes, exactly + teythoon: my fstab if you wanna take a look + http://pastebin.com/ef94JPhG + after I added /dev/hd0s3 to fstab along with its mountpoint, + and restarting the hurd, only then I did manage to use that partition + before doing so I tried pretty much anything involving + mounting the partition and setting the ext2fs translator for it, but it + kept dying + of course it was a ext2 filesystem + err, perhaps adding to fstab simply triggered an fsck at reboot? + nlightnfotis: might have been that you needed to reboot mach so + that it picks up the new partition table + youpi: I thought this was fixed, the partition reloading I mean? + that is needed, yes + let me check + youpi: it could be, though, to be honest, my hurd system + does an fsck all the time at boot + how do you manage to do that w/o rebooting for d-i? + (I don't remember whether device busy is detected) + teythoon: by making all translators go away, iirc + nlightnfotis: btw, you have ~/gcc_new as mountpoint in your + fstab, pretty sure that this cannot work, the path has to be absolute and + no ~ expansion is done + tbh it does work, and it's weird + nlightnfotis: it works b/c of the passive translator you set, + not b/c of the fstab entry + teythoon: should I change it? + probably, yes + Well, that is probably not used anywhere. + tschwinge: not yet but soon ;) + Isn't /etc/fstab only consulted for fsck. + atm yes + Anyway, it is definitely a very good idea to have a partition + separate from the rootfs for doing actual work. + I think I described that in one of the first GSoC coodridation + emails. In the long one. + teythoon: Oh it struck me now! Is it because tilde expansion + is only happening in bash, but /etc/fstab is read before bash is + initialized? + nlightnfotis: Instead of fumbling around with partitioning of + disk images, it may be easier in your KVM/QEMU setup to simply add a new + disk using -hdb [file] (or similar). + nlightnfotis: Basically, yes. + nlightnfotis: fstab is not related with bash in any way + anyway, it shouldn't matter now, it seems to be working, and + I wouldn't like fiddling around with it and messing it up now. I will + continue with resolving the gcc issues. + But /etc/fstab has its very own "language" (layout), so tilde + expansion will never be done there. + nlightnfotis: df -h ~/gcc_new/ + tschwinge: size 24G Used: 4.2G Avail 18G + OK, that's fine. + As you can see on + , GCC + will easily need some GiB. + tschwinge: I have some questions about GCC: out of curiosity + how much time does it take to compile it on your machine? Because + yesterday I tried a -w (suppress warnings) build and it seemed to take + forever + mind you the vm has 1536 ram available (I have read + somewhere that it can utilise such an amount) and the vm is KVM enabled + without disabling g++, it can easily take hours + nlightnfotis: The build error is unexpected, because I had + addressed that issue in a recent patch. :-) + nlightnfotis: This is wrong: »checking whether setcontext + clobbers TLS variables... [...] yes«. Please check your sources, that + they correspond to the current version of the upstream + tschwinge/t/hurd/go branch. + nlightnfotis: Quoting from that wiki page: »This takes up + around 3.5 GiB, and needs roughly 3.5 h on kepler.SCHWINGE and 15 h on + coulomb.SCHWINGE.« The latter is my Hurd machine. + That's however with Java and Ada enabled, and a full + three-stages bootstrap. + ah, right, there's java & ada too + tschwinge: git branch (in the repo): master, + *tschwinge/t/hurd/go + in debian they are built separately + What I asked you to do is configure »--disable-bootstrap + --enable-languages=go«. + So that should be a lot quicker. + tschwinge: oh yes, everytime I have tried to compile gcc I + have done with these configurations + But still a few hours perhaps. + that's what I did yesterday too. + OK, good. :-) + A bootstrap build is a good way to check the just-built GCC for + sanity, but we expect that it is fine, as we concentrate on the GCC Go + port. + the only "extra" configuration yesterday was my "-w" flag to + make, because those errors were actually triggered by -Werror + Let me read up what make -w does. ;-) + ah, yes, d/w I have read and understood what the bootstrap + build is. Seems like we don't need it atm + afaik it suppresses all warnings + youpi: gcj no more + the way gcc builds, it does convert (some) warnings to + errors + Hmm. -w, --print-directory Print a message containing the + working directory before and after other processing. + youpi: doko folded gcj and gdc into gcc-4.8 to "workaround" + Built-Using + nlightnfotis: Ah, that'S configure --enable-werror or something + like that. + pinotree: right + yep, and -w suppresses it + (from what I have understood) + nlightnfotis: Are you thinking about make -k? + Yeah, I guess. + let me see what -k does + youpi: (just to make builds even more lightweight, eh) + yeah, -k should do too, I shall try it + But: if gcc -Werror fails, even with make -k, the build will + not be able to come to a successful end, because that one complation + artefact that failed will be missing. + so I shall try again with -w (supressed warnings) + Configureing with --disable-werror (or similar) will "help" if + -Werror is the default, and the build fails due to that. + from what I have understood these "errors" are not something + critical: it's only that function prototypes for these functions are + missing + I have seen the code there, and even "default" gcc generated + prototypes (from the first usage of the function) should do, so I can't + understand why it might be a serious problem if I tell gcc to skip that + point + nlightnfotis: Ah, now I see. You don't mean make -w, but + rather gcc -w: »-w Inhibit all warning messages.« + But really, there shouldn't be such warnings/errors that make + the build fail. + yeah + nlightnfotis: In your GCC sources directory, what does this + tell: git rev-parse HEAD + And, is the checkout clean: git status + The latter will take some time. + git status takes an awful amount of time + last I checked + but git rev-parse HEAD + produces this result: + 91840dfb3942a8d241cc4f0e573e5a9956011532 + OK, that's correct. So probably some of the checked out files + are not in a pristine state? + I shall run a git clean and see. If that doesn't work too, + maybe I shall reclone the repository? + there's nothing foreign to the repo that I have added, only + lib gmp, lib mpc and lib mpfr (and they are in their own folders inside + my gcc working directory) + nlightnfotis: You shouldn't need to do the latter if you + instead run: apt-get build-dep gcc-4.8 + I remember having done that inside the Hurd, but it always + resulted in an error from what I can recall + let me check this out + yes + nlightnfotis: Whenever you use Git on Hurd, pass the --quiet + flag, to avoid the rare but possible corruption issue described on + + and . + tschwinge: Forgive me for that. I will set up an alias + immediately. + nlightnfotis: I don't know if an alias is possible, because -- + I think -- you'll need to do things like: git fetch --quiet + So pass --quiet to subcommands. + oh. ok. + nlightnfotis: What you can also do, is shut down your Hurd VM, + and mount the disk image on GNU/Linux (mount with offset to get the right + partition), and then run a diff -ru against a Git clone done on + GNU/Linux, and see whether there are any unexpected differences outside + of the .git/ directory. + sounds like a plan. I will check this out today then :) + tschwinge: if all else fails, then recloning the repo with + --quiet passed should work, right? + Yes, that's probably the most straight-forward check to do. + Heh, yes to both these questions. :-) + nlightnfotis: Oh, you don't even have to re-clone, but rather + re-check-out the branch. + I was thinking of recloning just to bring the whole + repository to a pristine state + So something like (inside the source directory): rm -rf ./* + (remove any files, but leave .* in place, in particular the .git/ + directory), followd by git checkout -f HEAD --quiet + nlightnfotis: But before doing that, please do the diff first, + so that we know (hopefully) where the erroneous build results were coming + from. + considering the Copyright assignment files, I have sent them + from day 1 (that is the 20th of June). I have not heard anything about + those documents to date (sadly) + what's worst is that although I have a reference number to + track those documents, their (greek postal office) tracking service sucks + so badly, that one day it's offline, the next it suggests it can't find + the object in their database, the next it says it is still in the local + post office + let me check it out now + still nothing from their online service + let me call them + tschwinge: I called the post office regarding the copyright + papers. They told me that the same day (the 20th of June) it left from + Herakleion, Crete to Athens and the same day it must have left the + country heading towards the US. They also told me it takes about 1 week + for it to arrive. + nlightnfotis: OK, so probably waiting at the FSF office to be + processed. Let's allow for some more time. After all, this is not + critical for your progress. -- cgit v1.2.3